graphene-elastic¶
Elasticsearch (DSL)/ OpenSearch (DSL) integration for Graphene.
Prerequisites¶
Graphene 2.x. Support for Graphene 1.x is not intended.
Python 3.6, 3.7, 3.8, 3.9 and 3.10. Support for Python 2 is not intended.
Elasticsearch 6.x, 7.x. Support for Elasticsearch 5.x is not intended.
OpenSearch 1.x, 2.x.
Main features and highlights¶
Implemented
ElasticsearchConnectionField
andElasticsearchObjectType
are the core classes to work withgraphene
.Pluggable backends for searching, filtering, ordering, etc. Don’t like existing ones? Override, extend or write your own.
Search backend.
Filter backend.
Ordering backend.
Pagination.
Highlighting backend.
Source filter backend.
Faceted search backend (including global aggregations).
Post filter backend.
Score filter backend.
Query string backend.
Simple query string backend.
See the Road-map for what’s yet planned to implemented.
Do you need a similar tool for Django REST Framework? Check django-elasticsearch-dsl-drf.
Demo¶
Check the live demo app (FastAPI + Graphene 2 + Elasticsearch 7) hosted on Heroku and bonsai.io.
Documentation¶
Documentation is available on Read the Docs.
Installation¶
Install latest stable version from PyPI:
pip install graphene-elastic
Or latest development version from GitHub:
pip install https://github.com/barseghyanartur/graphene-elastic/archive/master.zip
Note
Staring from version 0.8, the elasticsearch
and elasticsearch-dsl
packages are no longer installed by default. You must either install them
explicitly in your requirements or install as optional dependencies as
follows: pip install graphene-elastic[elasticsearch]
.
Alternatively, you can use opensearch-py
and opensearch-dsl
.
You would then need to install the opensearch-py
and opensearch-dsl
packages explicitly in your requirements or install them as optional
dependencies as follows: pip install graphene-elastic[opensearch]
.
Examples¶
Note
In the examples, we use elasticsearch_dsl
package for schema definition.
You can however use opensearch_dsl
or if you want to achieve
portability between Elasticsearch
and OpenSearch
, use anysearch
package. Read more here.
Install requirements¶
pip install -r requirements.txt
Populate sample data¶
The following command will create indexes for User
and Post
documents
and populate them with sample data:
./scripts/populate_elasticsearch_data.sh
Sample document definition¶
search_index/documents/post.py
See examples/search_index/documents/post.py for full example.
import datetime
from elasticsearch_dsl import (
Boolean,
Date,
Document,
InnerDoc,
Keyword,
Nested,
Text,
Integer,
)
class Comment(InnerDoc):
author = Text(fields={'raw': Keyword()})
content = Text(analyzer='snowball')
created_at = Date()
def age(self):
return datetime.datetime.now() - self.created_at
class Post(Document):
title = Text(
fields={'raw': Keyword()}
)
content = Text()
created_at = Date()
published = Boolean()
category = Text(
fields={'raw': Keyword()}
)
comments = Nested(Comment)
tags = Text(
analyzer=html_strip,
fields={'raw': Keyword(multi=True)},
multi=True
)
num_views = Integer()
class Index:
name = 'blog_post'
settings = {
'number_of_shards': 1,
'number_of_replicas': 1,
'blocks': {'read_only_allow_delete': None},
}
Sample apps¶
Sample Flask app¶
Run the sample Flask app:
./scripts/run_flask.sh
Open Flask graphiql client
http://127.0.0.1:8001/graphql
Sample Django app¶
Run the sample Django app:
./scripts/run_django.sh runserver
Open Django graphiql client
http://127.0.0.1:8000/graphql
ConnectionField example¶
ConnectionField is the most flexible and feature rich solution you have. It uses filter backends which you can tie to your needs the way you want in a declarative manner.
Sample schema definition
import graphene
from graphene_elastic import (
ElasticsearchObjectType,
ElasticsearchConnectionField,
)
from graphene_elastic.filter_backends import (
FilteringFilterBackend,
SearchFilterBackend,
HighlightFilterBackend,
OrderingFilterBackend,
DefaultOrderingFilterBackend,
)
from graphene_elastic.constants import (
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_EXCLUDE,
LOOKUP_QUERY_IN,
)
# Object type definition
class Post(ElasticsearchObjectType):
class Meta(object):
document = PostDocument
interfaces = (Node,)
filter_backends = [
FilteringFilterBackend,
SearchFilterBackend,
HighlightFilterBackend,
OrderingFilterBackend,
DefaultOrderingFilterBackend,
]
# For `FilteringFilterBackend` backend
filter_fields = {
# The dictionary key (in this case `title`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value could be simple or complex structure (in this case
# complex). The `field` key points to the `title.raw`, which
# is the field name in the Elasticsearch document
# (`PostDocument`). Since `lookups` key is provided, number
# of lookups is limited to the given set, while term is the
# default lookup (as specified in `default_lookup`).
'title': {
'field': 'title.raw',
# Available lookups
'lookups': [
LOOKUP_FILTER_TERM,
LOOKUP_FILTER_TERMS,
LOOKUP_FILTER_PREFIX,
LOOKUP_FILTER_WILDCARD,
LOOKUP_QUERY_IN,
LOOKUP_QUERY_EXCLUDE,
],
# Default lookup
'default_lookup': LOOKUP_FILTER_TERM,
},
# The dictionary key (in this case `category`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `category.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'category': 'category.raw',
# The dictionary key (in this case `tags`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `tags.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'tags': 'tags.raw',
# The dictionary key (in this case `num_views`) is the name of
# the corresponding GraphQL query argument. Since no lookups
# or default_lookup is provided, defaults are used (all lookups
# available, term is the default lookup). The dictionary value
# (in this case `num_views`) is the field name in the
# Elasticsearch document (`PostDocument`).
'num_views': 'num_views',
}
# For `SearchFilterBackend` backend
search_fields = {
'title': {'boost': 4},
'content': {'boost': 2},
'category': None,
}
# For `OrderingFilterBackend` backend
ordering_fields = {
# The dictionary key (in this case `tags`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `tags.raw`) is the field name in the
# Elasticsearch document (`PostDocument`).
'title': 'title.raw',
# The dictionary key (in this case `created_at`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `created_at`) is the field name in the
# Elasticsearch document (`PostDocument`).
'created_at': 'created_at',
# The dictionary key (in this case `num_views`) is the name of
# the corresponding GraphQL query argument. The dictionary
# value (in this case `num_views`) is the field name in the
# Elasticsearch document (`PostDocument`).
'num_views': 'num_views',
}
# For `DefaultOrderingFilterBackend` backend
ordering_defaults = (
'-num_views', # Field name in the Elasticsearch document
'title.raw', # Field name in the Elasticsearch document
)
# For `HighlightFilterBackend` backend
highlight_fields = {
'title': {
'enabled': True,
'options': {
'pre_tags': ["<b>"],
'post_tags': ["</b>"],
}
},
'content': {
'options': {
'fragment_size': 50,
'number_of_fragments': 3
}
},
'category': {},
}
# Query definition
class Query(graphene.ObjectType):
all_post_documents = ElasticsearchConnectionField(Post)
# Schema definition
schema = graphene.Schema(query=Query)
Filter¶
Sample queries¶
Since we didn’t specify any lookups on category, by default all lookups
are available and the default lookup would be term
. Note, that in the
{value:"Elastic"}
part, the value
stands for default lookup, whatever
it has been set to.
query PostsQuery {
allPostDocuments(filter:{category:{value:"Elastic"}}) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}
But, we could use another lookup (in example below - terms
). Note, that
in the {terms:["Elastic", "Python"]}
part, the terms
is the lookup
name.
query PostsQuery {
allPostDocuments(
filter:{category:{terms:["Elastic", "Python"]}}
) {
edges {
node {
id
title
category
content
createdAt
comments
}
}
}
}
Or apply a gt
(range
) query in addition to filtering:
{
allPostDocuments(filter:{
category:{term:"Python"},
numViews:{gt:"700"}
}) {
edges {
node {
category
title
comments
numViews
}
}
}
}
Implemented filter lookups¶
The following lookups are available:
contains
ends_with
(orendsWith
for camelCase)exclude
exists
gt
gte
in
is_null
(orisNull
for camelCase)lt
lte
prefix
range
starts_with
(orstartsWith
for camelCase)term
terms
wildcard
See dedicated documentation on filter lookups for more information.
Search¶
Search in all fields:
query {
allPostDocuments(
search:{query:"Release Box"}
) {
edges {
node {
category
title
content
}
}
}
}
Search in specific fields:
query {
allPostDocuments(
search:{
title:{value:"Release", boost:2},
content:{value:"Box"}
}
) {
edges {
node {
category
title
content
}
}
}
}
Ordering¶
Possible choices are ASC
and DESC
.
query {
allPostDocuments(
filter:{category:{term:"Photography"}},
ordering:{title:ASC}
) {
edges {
node {
category
title
content
numViews
tags
}
}
}
}
Pagination¶
The first
, last
, before
and after
arguments are supported.
By default number of results is limited to 100.
query {
allPostDocuments(first:12) {
pageInfo {
startCursor
endCursor
hasNextPage
hasPreviousPage
}
edges {
cursor
node {
category
title
content
numViews
}
}
}
}
Highlighting¶
Simply, list the fields you want to highlight. This works only in combination with search.
query {
allPostDocuments(
search:{content:{value:"alice"}, title:{value:"alice"}},
highlight:[category, content]
) {
edges {
node {
title
content
highlight
}
cursor
}
}
}
Road-map¶
Road-map and development plans.
This package is designed after django-elasticsearch-dsl-drf and is intended to offer similar functionality.
Lots of features are planned to be released in the upcoming Beta releases:
Suggester backend.
Nested backend.
Geo-spatial backend.
Filter lookup
geo_bounding_box
(orgeoBoundingBox
for camelCase).Filter lookup
geo_distance
(orgeoDistance
for camelCase).Filter lookup
geo_polygon
(orgeoPolygon
for camelCase).More-like-this backend.
Stay tuned or reach out if you want to help.
Testing¶
Project is covered with tests.
Testing with Docker¶
make docker-test
Running tests with virtualenv or tox¶
By defaults tests are executed against the Elasticsearch 7.x.
Run Elasticsearch 7.x with Docker
docker-compose up elasticsearch
Install test requirements
pip install -r requirements/test.txt
To test with all supported Python versions type:
tox
To test against specific environment, type:
tox -e py38-elastic7
To test just your working environment type:
./runtests.py
To run a single test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.py
To run a single test class in a given test module in your working environment type:
./runtests.py src/graphene_elastic/tests/test_filter_backend.py::FilterBackendElasticTestCase
Debugging¶
For development purposes, you could use the flask app (easy to debug). Standard
pdb
works (import pdb; pdb.set_trace()
). If ipdb
does not work
well for you, use ptpdb
.
Writing documentation¶
Keep the following hierarchy.
=====
title
=====
header
======
sub-header
----------
sub-sub-header
~~~~~~~~~~~~~~
sub-sub-sub-header
^^^^^^^^^^^^^^^^^^
sub-sub-sub-sub-header
++++++++++++++++++++++
sub-sub-sub-sub-sub-header
**************************
License¶
GPL-2.0-only OR LGPL-2.1-or-later
Support¶
For any security issues contact me at the e-mail given in the Author section. For overall issues, go to GitHub.
Project documentation¶
Contents:
- graphene-elastic
- Prerequisites
- Main features and highlights
- Demo
- Documentation
- Installation
- Examples
- Road-map
- Testing
- Debugging
- Writing documentation
- License
- Support
- Author
- Project documentation
- Concepts
- Quick start
- Search
- Filtering
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Post-filter backend
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Ordering
- Highlight
- Source
- Score
- Faceted search
- Query string backend
- Simple query string backend
- Pagination
- Custom filter backends
- Settings
- Running Elasticsearch
- FAQ
- Debugging
- Release history and notes
- graphene_elastic package
- Subpackages
- graphene_elastic.filter_backends package
- Subpackages
- graphene_elastic.filter_backends.faceted_search package
- graphene_elastic.filter_backends.filtering package
- graphene_elastic.filter_backends.highlight package
- graphene_elastic.filter_backends.ordering package
- graphene_elastic.filter_backends.post_filter package
- graphene_elastic.filter_backends.score package
- graphene_elastic.filter_backends.search package
- graphene_elastic.filter_backends.source package
- Submodules
- graphene_elastic.filter_backends.base module
- graphene_elastic.filter_backends.queries module
- Module contents
- Subpackages
- graphene_elastic.relay package
- graphene_elastic.tests package
- Submodules
- graphene_elastic.tests.base module
- graphene_elastic.tests.test_faceted_search_backend module
- graphene_elastic.tests.test_filter_backend module
- graphene_elastic.tests.test_highlight_backend module
- graphene_elastic.tests.test_ordering_backend module
- graphene_elastic.tests.test_pagination module
- graphene_elastic.tests.test_post_filter_backend module
- graphene_elastic.tests.test_query_string_backend module
- graphene_elastic.tests.test_score_backend module
- graphene_elastic.tests.test_search_backend module
- graphene_elastic.tests.test_simple_query_string_backend module
- graphene_elastic.tests.test_source_backend module
- graphene_elastic.tests.test_versions module
- Module contents
- graphene_elastic.types package
- graphene_elastic.filter_backends package
- Submodules
- graphene_elastic.advanced_types module
- graphene_elastic.arrayconnection module
- graphene_elastic.compat module
- graphene_elastic.constants module
- graphene_elastic.converter module
- graphene_elastic.enums module
- graphene_elastic.fields module
- graphene_elastic.logging module
- graphene_elastic.registry module
- graphene_elastic.settings module
- graphene_elastic.utils module
- graphene_elastic.versions module
- Module contents
- Subpackages
- Indices and tables
- Concepts
- Quick start
- Search
- Filtering
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Post-filter backend
- Filter lookups
- Filter lookup
contains
- Filter lookup
ends_with
- Filter lookup
exclude
- Filter lookup
exists
- Filter lookup
gt
- Filter lookup
gte
- Filter lookup
in
- Filter lookup
lt
- Filter lookup
lte
- Filter lookup
prefix
- Filter lookup
range
- Filter lookup
starts_with
- Filter lookup
term
- Filter lookup
terms
- Filter lookup
wildcard
- Filter lookup
- Filter lookups
- Ordering
- Highlight
- Source
- Score
- Faceted search
- Query string backend
- Simple query string backend
- Pagination
- Custom filter backends
- Settings
- Running Elasticsearch
- FAQ
- Debugging
- Release history and notes
- graphene_elastic package
- Subpackages
- graphene_elastic.filter_backends package
- Subpackages
- graphene_elastic.filter_backends.faceted_search package
- graphene_elastic.filter_backends.filtering package
- graphene_elastic.filter_backends.highlight package
- graphene_elastic.filter_backends.ordering package
- graphene_elastic.filter_backends.post_filter package
- graphene_elastic.filter_backends.score package
- graphene_elastic.filter_backends.search package
- graphene_elastic.filter_backends.source package
- Submodules
- graphene_elastic.filter_backends.base module
- graphene_elastic.filter_backends.queries module
- Module contents
- Subpackages
- graphene_elastic.relay package
- graphene_elastic.tests package
- Submodules
- graphene_elastic.tests.base module
- graphene_elastic.tests.test_faceted_search_backend module
- graphene_elastic.tests.test_filter_backend module
- graphene_elastic.tests.test_highlight_backend module
- graphene_elastic.tests.test_ordering_backend module
- graphene_elastic.tests.test_pagination module
- graphene_elastic.tests.test_post_filter_backend module
- graphene_elastic.tests.test_query_string_backend module
- graphene_elastic.tests.test_score_backend module
- graphene_elastic.tests.test_search_backend module
- graphene_elastic.tests.test_simple_query_string_backend module
- graphene_elastic.tests.test_source_backend module
- graphene_elastic.tests.test_versions module
- Module contents
- graphene_elastic.types package
- graphene_elastic.filter_backends package
- Submodules
- graphene_elastic.advanced_types module
- graphene_elastic.arrayconnection module
- graphene_elastic.compat module
- graphene_elastic.constants module
- graphene_elastic.converter module
- graphene_elastic.enums module
- graphene_elastic.fields module
- graphene_elastic.logging module
- graphene_elastic.registry module
- graphene_elastic.settings module
- graphene_elastic.utils module
- graphene_elastic.versions module
- Module contents
- Subpackages