Projects
Kolab:Winterfell
python-elasticsearch
Log In
Username
Password
We truncated the diff of some files because they were too big. If you want to see the full diff for every file,
click here
.
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
Expand all
Collapse all
Changes of Revision 2
View file
python-elasticsearch.spec
Changed
@@ -1,22 +1,55 @@ -%{!?__python2: %global __python2 %(which python)} -%{!?python2_sitelib: %global python2_sitelib %{python_sitelib}} - -%global tarball_name elasticsearch +%global tarball_name elasticsearch5 +%bcond_with python3 Name: python-elasticsearch -Version: 1.0.0 +Version: 5.5.3 Release: 1%{?dist} -Summary: Client for Elasticsearch +Summary: Client for Elasticsearch -Group: Development/Languages License: ASL 2.0 URL: https://github.com/elasticsearch/elasticsearch-py -Source0: https://pypi.python.org/packages/source/e/%{tarball_name}/%{tarball_name}-%{version}.tar.gz +Source0: https://pypi.io/packages/source/e/%{tarball_name}/%{tarball_name}-%{version}.tar.gz BuildArch: noarch -BuildRequires: python2-devel python-setuptools -Requires: python-thrift python-urllib3 -%description +BuildRequires: python2-devel +BuildRequires: python2-setuptools + +%if %{with python3} +BuildRequires: python3-devel +BuildRequires: python3-setuptools +# optional thrift not supported on Python 3 +%endif + +%global _description\ +Low level client for Elasticsearch. It's goal is to provide common ground\ +for all Elasticsearch-related code in Python. The client's features include:\ +\ +- Translating basic Python data types to and from json\ +- Configurable automatic discovery of cluster nodes\ +- Persistent connections\ +- Load balancing (with pluggable selection strategy) across all available nodes\ +- Failed connection penalization (time based - failed connections won't be\ + retried until a timeout is reached)\ +- Thread safety\ +- Pluggable architecture + +%description %_description + +%package -n python2-%{tarball_name} +Summary: %summary +Requires: python2-thrift +Requires: python2-urllib3 +%{?python_provide:%python_provide python2-%{tarball_name}} + +%description -n python2-%{tarball_name} %_description + +%if %{with python3} +%package -n python3-%{tarball_name} +Summary: Python 3 Client for Elasticsearch +License: ASL 2.0 +Requires: python3-urllib3 + +%description -n python3-%{tarball_name} Low level client for Elasticsearch. It's goal is to provide common ground for all Elasticsearch-related code in Python. The client's features include: @@ -28,27 +61,119 @@ retried until a timeout is reached) - Thread safety - Pluggable architecture +%endif %prep %setup -qn %{tarball_name}-%{version} rm -fr %{tarball_name}.egg-info %build -%{__python2} setup.py build +%py2_build +%if %{with python3} +%py3_build +%endif %install -rm -rf %{buildroot} -%{__python2} setup.py install -O1 --skip-build --root %{buildroot} +%if %{with python3} +%py3_install +%endif + +%py2_install + +%check +# * missing requirements for tests: python-pyaml +# * tests are not included in the tarball, we would have to add an extra +# source for them +# * https://github.com/elastic/elasticsearch-py/tree/master/test_elasticsearch +# also notes that "The tests also rely on a checkout of elasticsearch +# repository existing on the same level", and it looks like they also +# require a running elasticsearch server + +%files -n python2-%{tarball_name} +%{python2_sitelib}/%{tarball_name} +%{python2_sitelib}/%{tarball_name}-%{version}-py2.?.egg-info +%doc README +%license LICENSE -%files -%{python2_sitelib}/* -%doc README LICENSE +%if %{with python3} +%files -n python3-elasticsearch +%{python3_sitelib}/%{tarball_name} +%{python3_sitelib}/%{tarball_name}-%{version}-py3.?.egg-info +%doc README +%license LICENSE +%endif %changelog +* Tue Jun 19 2018 Miro Hrončok <mhroncok@redhat.com> - 2.4.0-7 +- Rebuilt for Python 3.7 + +* Fri Feb 09 2018 Fedora Release Engineering <releng@fedoraproject.org> - 2.4.0-6 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_28_Mass_Rebuild + +* Sat Aug 19 2017 Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> - 2.4.0-5 +- Python 2 binary package renamed to python2-elasticsearch + See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3 + +* Thu Jul 27 2017 Fedora Release Engineering <releng@fedoraproject.org> - 2.4.0-4 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_27_Mass_Rebuild + +* Sun Mar 12 2017 Piotr Popieluch <piotr1212@gmail.com> - 2.4.0-3 +- Update python thrift requires + +* Sat Feb 11 2017 Fedora Release Engineering <releng@fedoraproject.org> - 2.4.0-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_26_Mass_Rebuild + +* Wed Dec 28 2016 Adam Williamson <awilliam@redhat.com> - 2.4.0-1 +- Update to 2.4.0 (needed for latest elastic-curator) +- Document missing tests + +* Mon Dec 19 2016 Miro Hrončok <mhroncok@redhat.com> - 2.3.0-4 +- Rebuild for Python 3.6 + +* Tue Jul 19 2016 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 2.3.0-3 +- https://fedoraproject.org/wiki/Changes/Automatic_Provides_for_Python_RPM_Packages + +* Wed Jun 15 2016 Orion Poplawski <orion@cora.nwra.com> - 2.3.0-2 +- Drop %%py3dir and use new macros + +* Wed Jun 08 2016 Piotr Popieluch <piotr1212@gmail.com> - 2.3.0-1 +- Update to 2.3.0 + +* Wed Jun 08 2016 Piotr Popieluch <piotr1212@gmail.com> - 2.0.0-4 +- Readd python-rullib dependency, rpmlint Error needs to be ignored #1344121 + +* Thu Feb 04 2016 Fedora Release Engineering <releng@fedoraproject.org> - 2.0.0-3 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_24_Mass_Rebuild + +* Wed Nov 04 2015 Robert Kuska <rkuska@redhat.com> - 2.0.0-2 +- Rebuilt for Python3.5 rebuild + +* Wed Oct 14 2015 Daniel Bruno <dbruno@fedoraproject.org> - 2.0.0-1 +- Elasticsearch 2.0.0 version release + +* Tue Sep 22 2015 Daniel Bruno <dbruno@fedoraproject.org> - 1.7.0-1 +- Upgrade to 1.7.0 + +* Thu Jun 18 2015 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.4.0-3 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_23_Mass_Rebuild + +* Thu May 14 2015 Piotr Popieluch <piotr1212@gmail.com> - 1.4.0-2 +- Add python3 module +- Remove trailing whitespace +- Remove deprecated group tag +- Move license from %%doc to %%license +- Remove deprecated rm -rf buildroot +- fix rpmlint Error: explicit-lib-dependency python-urllib3 + +* Tue Apr 07 2015 Alan Pevec <apevec@fedoraproject.org> - 1.4.0-1 +- Upgrade to 1.4.0 version + +* Sat Jun 07 2014 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.0.0-2 +- Rebuilt for https://fedoraproject.org/wiki/Fedora_21_Mass_Rebuild + * Mon Apr 14 2014 Daniel Bruno <dbruno@fedoraproject.org> - 1.0.0-1 - Upgrade to 1.0.0 version * Tue Nov 26 2013 Daniel Bruno <dbruno@fedoraproject.org> - 0.4.3-1 - First RPM release -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch
Deleted
-(directory)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch.egg-info
Deleted
-(directory)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch.egg-info/PKG-INFO
Deleted
@@ -1,110 +0,0 @@ -Metadata-Version: 1.0 -Name: elasticsearch -Version: 1.0.0 -Summary: Python client for Elasticsearch -Home-page: https://github.com/elasticsearch/elasticsearch-py -Author: Honza Král -Author-email: honza.kral@gmail.com -License: Apache License, Version 2.0 -Description: Python Elasticsearch Client - =========================== - - Official low-level client for Elasticsearch. It's goal is to provide common - ground for all Elasticsearch-related code in Python; because of this it tries - to be opinion-free and very extendable. - - Versioning - ---------- - - There are two branches for development - `master` and `0.4`. Master branch is - used to track all the changes for Elasticsearch 1.0 and beyond whereas 0.4 - tracks Elasticsearch 0.90. - - Releases with major version 1 (1.X.Y) are to be used with Elasticsearch 1.0 and - later, 0.4 releases are meant to work with Elasticsearch 0.90.X. - - - Installation - ------------ - - Install the `elasticsearch` package with `pip - <https://pypi.python.org/pypi/elasticsearch>`_:: - - pip install elasticsearch - - - Example use - ----------- - - Simple use-case:: - - >>> from datetime import datetime - >>> from elasticsearch import Elasticsearch - - # by default we connect to localhost:9200 - >>> es = Elasticsearch() - - # datetimes will be serialized - >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()}) - {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True} - - # but not deserialized - >>> es.get(index="my-index", doc_type="test-type", id=42)['_source'] - {u'any': u'data', u'timestamp': u'2013-05-12T19:45:31.804229'} - - `Full documentation`_. - - .. _Full documentation: http://elasticsearch-py.rtfd.org/ - - - Features - -------- - - The client's features include: - - * translating basic Python data types to and from json (datetimes are not - decoded for performance reasons) - * configurable automatic discovery of cluster nodes - * persistent connections - * load balancing (with pluggable selection strategy) across all availible nodes - * failed connection penalization (time based - failed connections won't be - retried until a timeout is reached) - * thread safety - * pluggable architecture - - - License - ------- - - Copyright 2013 Elasticsearch - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. - - Build status - ------------ - - .. image:: https://secure.travis-ci.org/elasticsearch/elasticsearch-py.png - :target: http://travis-ci.org/#!/elasticsearch/elasticsearch-py - - .. image:: https://coveralls.io/repos/elasticsearch/elasticsearch-py/badge.png?branch=master - :target: https://coveralls.io/r/elasticsearch/elasticsearch-py -Platform: UNKNOWN -Classifier: Development Status :: 4 - Beta -Classifier: License :: OSI Approved :: Apache Software License -Classifier: Intended Audience :: Developers -Classifier: Operating System :: OS Independent -Classifier: Programming Language :: Python -Classifier: Programming Language :: Python :: 2 -Classifier: Programming Language :: Python :: 3 -Classifier: Programming Language :: Python :: Implementation :: CPython -Classifier: Programming Language :: Python :: Implementation :: PyPy
View file
elasticsearch-1.0.0.tar.gz/elasticsearch.egg-info/SOURCES.txt
Deleted
@@ -1,48 +0,0 @@ -AUTHORS -CONTRIBUTING.md -Changelog.rst -LICENSE -MANIFEST.in -README -README.rst -setup.cfg -setup.py -tox.ini -docs/Changelog.rst -docs/Makefile -docs/api.rst -docs/conf.py -docs/connection.rst -docs/helpers.rst -docs/index.rst -docs/transports.rst -elasticsearch/__init__.py -elasticsearch/connection_pool.py -elasticsearch/exceptions.py -elasticsearch/helpers.py -elasticsearch/serializer.py -elasticsearch/transport.py -elasticsearch.egg-info/PKG-INFO -elasticsearch.egg-info/SOURCES.txt -elasticsearch.egg-info/dependency_links.txt -elasticsearch.egg-info/requires.txt -elasticsearch.egg-info/top_level.txt -elasticsearch/client/__init__.py -elasticsearch/client/cat.py -elasticsearch/client/cluster.py -elasticsearch/client/indices.py -elasticsearch/client/nodes.py -elasticsearch/client/snapshot.py -elasticsearch/client/utils.py -elasticsearch/connection/__init__.py -elasticsearch/connection/base.py -elasticsearch/connection/http_requests.py -elasticsearch/connection/http_urllib3.py -elasticsearch/connection/memcached.py -elasticsearch/connection/pooling.py -elasticsearch/connection/thrift.py -elasticsearch/connection/esthrift/Rest-remote -elasticsearch/connection/esthrift/Rest.py -elasticsearch/connection/esthrift/__init__.py -elasticsearch/connection/esthrift/constants.py -elasticsearch/connection/esthrift/ttypes.py \ No newline at end of file
View file
elasticsearch-1.0.0.tar.gz/elasticsearch.egg-info/requires.txt
Deleted
@@ -1,1 +0,0 @@ -urllib3>=1.5, <2.0 \ No newline at end of file
View file
elasticsearch-1.0.0.tar.gz/elasticsearch.egg-info/top_level.txt
Deleted
@@ -1,1 +0,0 @@ -elasticsearch
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/__init__.py
Deleted
@@ -1,15 +0,0 @@ -from __future__ import absolute_import - -VERSION = (1, 0, 0) -__version__ = VERSION -__versionstr__ = '.'.join(map(str, VERSION)) - -from elasticsearch.client import Elasticsearch -from elasticsearch.transport import Transport -from elasticsearch.connection_pool import ConnectionPool, ConnectionSelector, \ - RoundRobinSelector -from elasticsearch.serializer import JSONSerializer -from elasticsearch.connection import Connection, RequestsHttpConnection, \ - Urllib3HttpConnection, MemcachedConnection, ThriftConnection -from elasticsearch.exceptions import * -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client
Deleted
-(directory)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/__init__.py
Deleted
@@ -1,858 +0,0 @@ -import weakref -import logging - -from ..transport import Transport -from ..exceptions import NotFoundError, TransportError -from .indices import IndicesClient -from .cluster import ClusterClient -from .cat import CatClient -from .nodes import NodesClient -from .snapshot import SnapshotClient -from .utils import query_params, _make_path - -logger = logging.getLogger('elasticsearch') - -def _normalize_hosts(hosts): - """ - Helper function to transform hosts argument to - :class:`~elasticsearch.Elasticsearch` to a list of dicts. - """ - # if hosts are empty, just defer to defaults down the line - if hosts is None: - return [{}] - - # passed in just one string - if isinstance(hosts, (type(''), type(u''))): - hosts = [hosts] - - out = [] - # normalize hosts to dicts - for i, host in enumerate(hosts): - if isinstance(host, (type(''), type(u''))): - host = host.strip('/') - # remove schema information - if '://' in host: - logger.warning( - "List of nodes should not include schema information (http://): %r.", - host - ) - host = host[host.index('://') + 3:] - - h = {"host": host} - if ':' in host: - # TODO: detect auth urls - host, port = host.rsplit(':', 1) - if port.isdigit(): - port = int(port) - h = {"host": host, "port": port} - out.append(h) - else: - out.append(host) - return out - - -class Elasticsearch(object): - """ - Elasticsearch low-level client. Provides a straightforward mapping from - Python to ES REST endpoints. - - The instance has attributes `cat`, `cluster`, `indices`, `nodes` and - `snapshot` that provide access to instances of - :class:`~elasticsearch.client.CatClient`, - :class:`~elasticsearch.client.ClusterClient`, - :class:`~elasticsearch.client.IndicesClient`, - :class:`~elasticsearch.client.NodesClient` and - :class:`~elasticsearch.client.SnapshotClient` respectively. This is the - preferred (and only supported) way to get access to those classes and their - methods. - - Some examples:: - - # create connection to localhost using the ThriftConnection and it's - # default port (9500) - es = Elasticsearch(connection_class=ThriftConnection) - - # create connection that will automatically inspect the cluster to get - # the list of active nodes. Start with nodes 'esnode1' and 'esnode2' - es = Elasticsearch( - ['esnode1', 'esnode2'], - # sniff before doing anything - sniff_on_start=True, - # refresh nodes after a node fails to respond - sniff_on_connection_fail=True, - # and also every 60 seconds - sniffer_timeout=60 - ) - - # connect to localhost directly and another node using SSL on port 443 - # and an url_prefix - es = Elasticsearch([ - {'host': 'localhost'}, - {'host': 'othernode', 'port': 443, 'url_prefix': 'es', 'use_ssl': True}, - ]) - """ - def __init__(self, hosts=None, transport_class=Transport, **kwargs): - """ - :arg hosts: list of nodes we should connect to. Node should be a - dictionary ({"host": "localhost", "port": 9200}), the entire dictionary - will be passed to the :class:`~elasticsearch.Connection` class as - kwargs, or a string in the format of ``host[:port]`` which will be - translated to a dictionary automatically. If no value is given the - :class:`~elasticsearch.Connection` class defaults will be used. - - :arg transport_class: :class:`~elasticsearch.Transport` subclass to use. - - :arg kwargs: any additional arguments will be passed on to the - :class:`~elasticsearch.Transport` class and, subsequently, to the - :class:`~elasticsearch.Connection` instances. - """ - self.transport = transport_class(_normalize_hosts(hosts), **kwargs) - - # namespaced clients for compatibility with API names - # use weakref to make GC's work a little easier - self.indices = IndicesClient(weakref.proxy(self)) - self.cluster = ClusterClient(weakref.proxy(self)) - self.cat = CatClient(weakref.proxy(self)) - self.nodes = NodesClient(weakref.proxy(self)) - self.snapshot = SnapshotClient(weakref.proxy(self)) - - def __repr__(self): - try: - # get a lost of all connections - cons = self.transport.hosts - # truncate to 10 if there are too many - if len(cons) > 5: - cons = cons[:5] + ['...'] - return '<Elasticsearch(%r)>' % cons - except: - # probably operating on custom transport and connection_pool, ignore - return super(Elasticsearch, self).__repr__() - - def _bulk_body(self, body): - # if not passed in a string, serialize items and join by newline - if not isinstance(body, (type(''), type(u''))): - body = '\n'.join(map(self.transport.serializer.dumps, body)) - - # bulk body must end with a newline - if not body.endswith('\n'): - body += '\n' - - return body - - @query_params() - def ping(self, params=None): - """ Returns True if the cluster is up, False otherwise. """ - try: - self.transport.perform_request('HEAD', '/', params=params) - except TransportError: - return False - return True - - @query_params() - def info(self, params=None): - """ Get the basic info from the current cluster. """ - _, data = self.transport.perform_request('GET', '/', params=params) - return data - - @query_params('consistency', 'parent', 'percolate', 'refresh', - 'replication', 'routing', 'timeout', 'timestamp', 'ttl', 'version', 'version_type') - def create(self, index, doc_type, body, id=None, params=None): - """ - Adds a typed JSON document in a specific index, making it searchable. - Behind the scenes this method calls index(..., op_type='create') - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html>`_ - - :arg index: The name of the index - :arg doc_type: The type of the document - :arg id: Document ID - :arg body: The document - :arg consistency: Explicit write consistency setting for the operation - :arg id: Specific document ID (when the POST method is used) - :arg parent: ID of the parent document - :arg percolate: Percolator queries to execute while indexing the document - :arg refresh: Refresh the index after performing the operation - :arg replication: Specific replication type (default: sync) - :arg routing: Specific routing value - :arg timeout: Explicit operation timeout - :arg timestamp: Explicit timestamp for the document - :arg ttl: Expiration time for the document - :arg version: Explicit version number for concurrency control - :arg version_type: Specific version type - """ - return self.index(index, doc_type, body, id=id, params=params, op_type='create') - - @query_params('consistency', 'op_type', 'parent', 'refresh', - 'replication', 'routing', 'timeout', 'timestamp', 'ttl', 'version', 'version_type') - def index(self, index, doc_type, body, id=None, params=None): - """ - Adds or updates a typed JSON document in a specific index, making it searchable. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html>`_ - - :arg index: The name of the index - :arg doc_type: The type of the document - :arg body: The document - :arg id: Document ID - :arg consistency: Explicit write consistency setting for the operation - :arg op_type: Explicit operation type (default: index) - :arg parent: ID of the parent document - :arg refresh: Refresh the index after performing the operation - :arg replication: Specific replication type (default: sync)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/cat.py
Deleted
@@ -1,234 +0,0 @@ -from .utils import NamespacedClient, query_params, _make_path - -class CatClient(NamespacedClient): - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def aliases(self, name=None, params=None): - """ - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-aliases.html>`_ - - :arg name: A comma-separated list of alias names to return - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'aliases', name), params=params) - return data - - @query_params('bytes', 'h', 'help', 'local', 'master_timeout', 'v') - def allocation(self, node_id=None, params=None): - """ - Allocation provides a snapshot of how shards have located around the - cluster and the state of disk usage. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-allocation.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to limit the - returned information - :arg bytes: The unit in which to display byte values - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'allocation', node_id), params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def count(self, index=None, params=None): - """ - Count provides quick access to the document count of the entire cluster, - or individual indices. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-count.html>`_ - - :arg index: A comma-separated list of index names to limit the returned - information - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'count', index), params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'ts', 'v') - def health(self, params=None): - """ - health is a terse, one-line representation of the same information from - :meth:`~elasticsearch.client.cluster.ClusterClient.health` API - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-health.html>`_ - - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg ts: Set to false to disable timestamping, default True - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', '/_cat/health', - params=params) - return data - - @query_params('help') - def help(self, params=None): - """ - A simple help for the cat api. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat.html>`_ - - :arg help: Return help information, default False - """ - _, data = self.transport.perform_request('GET', '/_cat', params=params) - return data - - @query_params('bytes', 'h', 'help', 'local', 'master_timeout', 'pri', 'v') - def indices(self, index=None, params=None): - """ - The indices command provides a cross-section of each index. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-indices.html>`_ - - :arg index: A comma-separated list of index names to limit the returned - information - :arg bytes: The unit in which to display byte values - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg pri: Set to true to return stats only for primary shards, default - False - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'indices', index), params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def master(self, params=None): - """ - Displays the master's node ID, bound IP address, and node name. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-master.html>`_ - - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', '/_cat/master', - params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def nodes(self, params=None): - """ - The nodes command shows the cluster topology. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-nodes.html>`_ - - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', '/_cat/nodes', - params=params) - return data - - @query_params('bytes', 'h', 'help', 'local', 'master_timeout', 'v') - def recovery(self, index=None, params=None): - """ - recovery is a view of shard replication. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-recovery.html>`_ - - :arg index: A comma-separated list of index names to limit the returned - information - :arg bytes: The unit in which to display byte values - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'recovery', index), params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def shards(self, index=None, params=None): - """ - The shards command is the detailed view of what nodes contain which shards. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/cat-shards.html>`_ - - :arg index: A comma-separated list of index names to limit the returned - information - :arg h: Comma-separated list of column names to display - :arg help: Return help information, default False - :arg local: Return local information, do not retrieve the state from - master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg v: Verbose mode. Display column headers, default False - """ - _, data = self.transport.perform_request('GET', _make_path('_cat', - 'shards', index), params=params) - return data - - @query_params('h', 'help', 'local', 'master_timeout', 'v') - def pending_tasks(self, params=None): - """ - pending_tasks provides the same information as the
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/cluster.py
Deleted
@@ -1,124 +0,0 @@ -from .utils import NamespacedClient, query_params, _make_path - -class ClusterClient(NamespacedClient): - @query_params('level', 'local', 'master_timeout', 'timeout', - 'wait_for_active_shards', 'wait_for_nodes', 'wait_for_relocating_shards', - 'wait_for_status') - def health(self, index=None, params=None): - """ - Get a very simple status on the health of the cluster. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-health.html>`_ - - :arg index: Limit the information returned to a specific index - :arg level: Specify the level of detail for returned information, default u'cluster' - :arg local: Return local information, do not retrieve the state from master node (default: false) - :arg master_timeout: Explicit operation timeout for connection to master node - :arg timeout: Explicit operation timeout - :arg wait_for_active_shards: Wait until the specified number of shards is active - :arg wait_for_nodes: Wait until the specified number of nodes is available - :arg wait_for_relocating_shards: Wait until the specified number of relocating shards is finished - :arg wait_for_status: Wait until cluster is in a specific state, default None - """ - _, data = self.transport.perform_request('GET', _make_path('_cluster', 'health', index), - params=params) - return data - - @query_params('local', 'master_timeout') - def pending_tasks(self, params=None): - """ - The pending cluster tasks API returns a list of any cluster-level - changes (e.g. create index, update mapping, allocate or fail shard) - which have not yet been executed. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-pending.html>`_ - - :arg local: Return local information, do not retrieve the state from master node (default: false) - :arg master_timeout: Specify timeout for connection to master - """ - _, data = self.transport.perform_request('GET', '/_cluster/pending_tasks', - params=params) - return data - - @query_params('index_templates', 'local', 'master_timeout', 'flat_settings') - def state(self, metric=None, index=None, params=None): - """ - Get a comprehensive state information of the whole cluster. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-state.html>`_ - - :arg metric: Limit the information returned to the specified metrics. - Possible values: "_all", "blocks", "index_templates", "metadata", - "nodes", "routing_table" - :arg index: A comma-separated list of index names; use `_all` or empty - string to perform the operation on all indices - :arg index_templates: A comma separated list to return specific index - templates when returning metadata. - :arg local: Return local information, do not retrieve the state from master node (default: false) - :arg master_timeout: Specify timeout for connection to master - :arg flat_settings: Return settings in flat format (default: false) - """ - if index and not metric: - metric = '_all' - _, data = self.transport.perform_request('GET', _make_path('_cluster', 'state', metric, index), params=params) - return data - - @query_params('flat_settings', 'human') - def stats(self, node_id=None, params=None): - """ - The Cluster Stats API allows to retrieve statistics from a cluster wide - perspective. The API returns basic index metrics and information about - the current nodes that form the cluster. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-stats.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to limit the - returned information; use `_local` to return information from the node - you're connecting to, leave empty to get information from all nodes - :arg flat_settings: Return settings in flat format (default: false) - :arg human: Whether to return time and byte values in human-readable format. - - """ - url = '/_cluster/stats' - if node_id: - url = _make_path('_cluster/stats/nodes', node_id) - _, data = self.transport.perform_request('GET', url, params=params) - return data - - @query_params('dry_run', 'filter_metadata', 'master_timeout', 'timeout') - def reroute(self, body=None, params=None): - """ - Explicitly execute a cluster reroute allocation command including specific commands. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-reroute.html>`_ - - :arg body: The definition of `commands` to perform (`move`, `cancel`, `allocate`) - :arg dry_run: Simulate the operation only and return the resulting state - :arg filter_metadata: Don't return cluster state metadata (default: false) - :arg master_timeout: Explicit operation timeout for connection to master node - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('POST', '/_cluster/reroute', params=params, body=body) - return data - - @query_params('flat_settings', 'master_timeout', 'timeout') - def get_settings(self, params=None): - """ - Get cluster settings. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html>`_ - - :arg flat_settings: Return settings in flat format (default: false) - :arg master_timeout: Explicit operation timeout for connection to master node - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('GET', '/_cluster/settings', params=params) - return data - - @query_params('flat_settings') - def put_settings(self, body, params=None): - """ - Update cluster wide specific settings. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html>`_ - - :arg body: The settings to be updated. Can be either `transient` or - `persistent` (survives cluster restart). - :arg flat_settings: Return settings in flat format (default: false) - """ - _, data = self.transport.perform_request('PUT', '/_cluster/settings', params=params, body=body) - return data -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/indices.py
Deleted
@@ -1,761 +0,0 @@ -from .utils import NamespacedClient, query_params, _make_path -from ..exceptions import NotFoundError - -class IndicesClient(NamespacedClient): - @query_params('analyzer', 'field', 'filters', 'format', 'index', - 'prefer_local', 'text', 'tokenizer') - def analyze(self, index=None, body=None, params=None): - """ - Perform the analysis process on a text and return the tokens breakdown of the text. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-analyze.html>`_ - - :arg index: The name of the index to scope the operation - :arg body: The text on which the analysis should be performed - :arg analyzer: The name of the analyzer to use - :arg field: Use the analyzer configured for this field (instead of - passing the analyzer name) - :arg filters: A comma-separated list of filters to use for the analysis - :arg format: Format of the output, default u'detailed' - :arg index: The name of the index to scope the operation - :arg prefer_local: With `true`, specify that a local shard should be - used if available, with `false`, use a random shard (default: true) - :arg text: The text on which the analysis should be performed (when - request body is not used) - :arg tokenizer: The name of the tokenizer to use for the analysis - """ - _, data = self.transport.perform_request('GET', _make_path(index, '_analyze'), - params=params, body=body) - return data - - @query_params('allow_no_indices', 'expand_wildcards', 'ignore_indices', - 'ignore_unavailable', 'force') - def refresh(self, index=None, params=None): - """ - Explicitly refresh one or more index, making all operations performed - since the last refresh available for search. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-refresh.html>`_ - - :arg index: A comma-separated list of index names; use `_all` or empty - string to perform the operation on all indices - :arg allow_no_indices: Whether to ignore if a wildcard indices - expression resolves into no concrete indices. (This includes `_all` string or - when no indices have been specified) - :arg expand_wildcards: Whether to expand wildcard expression to concrete indices - that are open, closed or both. - :arg ignore_indices: When performed on multiple indices, allows to - ignore `missing` ones, default u'none' - :arg ignore_unavailable: Whether specified concrete indices should be ignored - when unavailable (missing or closed) - :arg force: Force a refresh even if not required - """ - _, data = self.transport.perform_request('POST', _make_path(index, '_refresh'), - params=params) - return data - - @query_params('force', 'full', 'allow_no_indices', 'expand_wildcards', - 'ignore_indices', 'ignore_unavailable') - def flush(self, index=None, params=None): - """ - Explicitly flush one or more indices. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-flush.html>`_ - - :arg index: A comma-separated list of index names; use `_all` or empty - string for all indices - :arg force: Whether a flush should be forced even if it is not - necessarily needed ie. if no changes will be committed to the index. - :arg full: If set to true a new index writer is created and settings - that have been changed related to the index writer will be refreshed. - :arg allow_no_indices: Whether to ignore if a wildcard indices - expression resolves into no concrete indices. (This includes `_all` string or - when no indices have been specified) - :arg expand_wildcards: Whether to expand wildcard expression to concrete indices - that are open, closed or both. - :arg ignore_indices: When performed on multiple indices, allows to - ignore `missing` ones (default: none) - :arg ignore_unavailable: Whether specified concrete indices should be ignored - when unavailable (missing or closed) - """ - _, data = self.transport.perform_request('POST', _make_path(index, '_flush'), - params=params) - return data - - @query_params('timeout', 'master_timeout') - def create(self, index, body=None, params=None): - """ - Create an index in Elasticsearch. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html>`_ - - :arg index: The name of the index - :arg body: The configuration for the index (`settings` and `mappings`) - :arg master_timeout: Specify timeout for connection to master - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('PUT', _make_path(index), - params=params, body=body) - return data - - @query_params('timeout', 'master_timeout' 'allow_no_indices', 'expand_wildcards', - 'ignore_unavailable') - def open(self, index, params=None): - """ - Open a closed index to make it available for search. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-open-close.html>`_ - - :arg index: The name of the index - :arg master_timeout: Specify timeout for connection to master - :arg timeout: Explicit operation timeout - :arg allow_no_indices: Whether to ignore if a wildcard indices - expression resolves into no concrete indices. (This includes `_all` string or - when no indices have been specified) - :arg expand_wildcards: Whether to expand wildcard expression to concrete indices - that are open, closed or both. - :arg ignore_unavailable: Whether specified concrete indices should be ignored - when unavailable (missing or closed) - """ - _, data = self.transport.perform_request('POST', _make_path(index, '_open'), - params=params) - return data - - @query_params('timeout', 'master_timeout') - def close(self, index, params=None): - """ - Close an index to remove it's overhead from the cluster. Closed index - is blocked for read/write operations. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-open-close.html>`_ - - :arg index: A comma-separated list of indices to delete; use `_all` or - '*' to delete all indices - :arg master_timeout: Specify timeout for connection to master - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('POST', _make_path(index, '_close'), - params=params) - return data - - @query_params('timeout', 'master_timeout') - def delete(self, index, params=None): - """ - Delete an index in Elasticsearch - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-delete-index.html>`_ - - :arg index: A comma-separated list of indices to delete; use `_all` or - '*' to delete all indices - :arg master_timeout: Specify timeout for connection to master - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('DELETE', _make_path(index), - params=params) - return data - - @query_params('local') - def exists(self, index, params=None): - """ - Return a boolean indicating whether given index exists. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-indices-exists.html>`_ - - :arg index: A list of indices to check - :arg local: Return local information, do not retrieve the state from - master node (default: false) - """ - try: - self.transport.perform_request('HEAD', _make_path(index), params=params) - except NotFoundError: - return False - return True - - @query_params('allow_no_indices', 'expand_wildcards', 'ignore_indices', 'ignore_unavailable', - 'local') - def exists_type(self, index, doc_type, params=None): - """ - Check if a type/types exists in an index/indices. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-types-exists.html>`_ - - :arg index: A comma-separated list of index names; use `_all` to check - the types across all indices - :arg doc_type: A comma-separated list of document types to check - :arg allow_no_indices: Whether to ignore if a wildcard indices - expression resolves into no concrete indices. (This includes `_all` string or - when no indices have been specified) - :arg expand_wildcards: Whether to expand wildcard expression to concrete indices - that are open, closed or both. - :arg ignore_indices: When performed on multiple indices, allows to - ignore `missing` ones (default: none) - :arg ignore_unavailable: Whether specified concrete indices should be ignored - when unavailable (missing or closed) - :arg local: Return local information, do not retrieve the state from - master node (default: false) - """ - try: - self.transport.perform_request('HEAD', _make_path(index, doc_type), params=params) - except NotFoundError: - return False - return True - - @query_params('allow_no_indices', 'expand_wildcards', 'ignore_indices', 'ignore_unavailable') - def snapshot_index(self, index=None, params=None): - """ - Explicitly perform a snapshot through the gateway of one or more indices (backup them). - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-gateway-snapshot.html>`_ -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/nodes.py
Deleted
@@ -1,106 +0,0 @@ -from .utils import NamespacedClient, query_params, _make_path - -class NodesClient(NamespacedClient): - @query_params('flat_settings', 'human') - def info(self, node_id=None, metric=None, params=None): - """ - The cluster nodes info API allows to retrieve one or more (or all) of - the cluster nodes information. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-info.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to limit the - returned information; use `_local` to return information from the - node you're connecting to, leave empty to get information from all - nodes - :arg metric: A comma-separated list of metrics you wish returned. Leave - empty to return all. Choices are "settings", "os", "process", - "jvm", "thread_pool", "network", "transport", "http", "plugin" - :arg flat_settings: Return settings in flat format (default: false) - :arg human: Whether to return time and byte values in human-readable - format., default False - """ - _, data = self.transport.perform_request('GET', _make_path('_nodes', - node_id, metric), params=params) - return data - - @query_params('delay', 'exit') - def shutdown(self, node_id=None, params=None): - """ - The nodes shutdown API allows to shutdown one or more (or all) nodes in - the cluster. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-shutdown.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to perform the - operation on; use `_local` to perform the operation on the node - you're connected to, leave empty to perform the operation on all - nodes - :arg delay: Set the delay for the operation (default: 1s) - :arg exit: Exit the JVM as well (default: true) - """ - _, data = self.transport.perform_request('POST', _make_path('_cluster', - 'nodes', node_id, '_shutdown'), params=params) - return data - - @query_params('completion_fields', 'fielddata_fields', 'fields', 'groups', 'human', 'level', 'types') - def stats(self, node_id=None, metric=None, index_metric=None, params=None): - """ - The cluster nodes stats API allows to retrieve one or more (or all) of - the cluster nodes statistics. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to limit the - returned information; use `_local` to return information from the - node you're connecting to, leave empty to get information from all - nodes - :arg metric: Limit the information returned to the specified metrics. - Possible options are: "_all", "breaker", "fs", "http", "indices", - "jvm", "network", "os", "process", "thread_pool", "transport" - :arg index_metric: Limit the information returned for `indices` metric - to the specific index metrics. Isn't used if `indices` (or `all`) - metric isn't specified. Possible options are: "_all", "completion", - "docs", "fielddata", "filter_cache", "flush", "get", "id_cache", - "indexing", "merge", "percolate", "refresh", "search", "segments", - "store", "warmer" - :arg completion_fields: A comma-separated list of fields for `fielddata` - and `suggest` index metric (supports wildcards) - :arg fielddata_fields: A comma-separated list of fields for `fielddata` - index metric (supports wildcards) - :arg fields: A comma-separated list of fields for `fielddata` and - `completion` index metric (supports wildcards) - :arg groups: A comma-separated list of search groups for `search` index - metric - :arg human: Whether to return time and byte values in human-readable - format., default False - :arg level: Return indices stats aggregated at node, index or shard - level, default 'node' - :arg types: A comma-separated list of document types for the `indexing` - index metric - """ - _, data = self.transport.perform_request('GET', _make_path('_nodes', - node_id, 'stats', metric, index_metric), params=params) - return data - - @query_params('type_', 'interval', 'snapshots', 'threads') - def hot_threads(self, node_id=None, params=None): - """ - An API allowing to get the current hot threads on each node in the cluster. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html>`_ - - :arg node_id: A comma-separated list of node IDs or names to limit the - returned information; use `_local` to return information from the - node you're connecting to, leave empty to get information from all - nodes - :arg type_: The type to sample (default: cpu) - :arg interval: The interval for the second sampling of threads - :arg snapshots: Number of samples of thread stacktrace (default: 10) - :arg threads: Specify the number of threads to provide information for - (default: 3) - """ - # avoid python reserved words - if params and 'type_' in params: - params['type'] = params.pop('type_') - _, data = self.transport.perform_request('GET', _make_path('_nodes', - node_id, 'hot_threads'), params=params) - return data - -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/snapshot.py
Deleted
@@ -1,117 +0,0 @@ -from .utils import NamespacedClient, query_params, _make_path - -class SnapshotClient(NamespacedClient): - @query_params('master_timeout', 'wait_for_completion') - def create(self, repository, snapshot, body=None, params=None): - """ - Create a snapshot in repository - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A repository name - :arg snapshot: A snapshot name - :arg body: The snapshot definition - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg wait_for_completion: Should this request wait until the operation - has completed before returning, default False - """ - _, data = self.transport.perform_request('PUT', _make_path('_snapshot', - repository, snapshot), params=params, body=body) - return data - - @query_params('master_timeout') - def delete(self, repository, snapshot, params=None): - """ - Deletes a snapshot from a repository. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A repository name - :arg snapshot: A snapshot name - :arg master_timeout: Explicit operation timeout for connection to master - node - """ - _, data = self.transport.perform_request('DELETE', - _make_path('_snapshot', repository, snapshot), params=params) - return data - - @query_params('master_timeout') - def get(self, repository, snapshot, params=None): - """ - Retrieve information about a snapshot. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A comma-separated list of repository names - :arg snapshot: A comma-separated list of snapshot names - :arg master_timeout: Explicit operation timeout for connection to master - node - """ - _, data = self.transport.perform_request('GET', _make_path('_snapshot', - repository, snapshot), params=params) - return data - - @query_params('master_timeout', 'timeout') - def delete_repository(self, repository, params=None): - """ - Removes a shared file system repository. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A comma-separated list of repository names - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('DELETE', - _make_path('_snapshot', repository), params=params) - return data - - @query_params('local', 'master_timeout') - def get_repository(self, repository=None, params=None): - """ - Return information about registered repositories. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A comma-separated list of repository names - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg local: Return local information, do not retrieve the state from - master node (default: false) - """ - _, data = self.transport.perform_request('GET', _make_path('_snapshot', - repository), params=params) - return data - - @query_params('master_timeout', 'timeout') - def create_repository(self, repository, body, params=None): - """ - Registers a shared file system repository. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A repository name - :arg body: The repository definition - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg timeout: Explicit operation timeout - """ - _, data = self.transport.perform_request('PUT', _make_path('_snapshot', - repository), params=params, body=body) - return data - - @query_params('master_timeout', 'wait_for_completion') - def restore(self, repository, snapshot, body=None, params=None): - """ - Restore a snapshot. - `<http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>`_ - - :arg repository: A repository name - :arg snapshot: A snapshot name - :arg body: Details of what to restore - :arg master_timeout: Explicit operation timeout for connection to master - node - :arg wait_for_completion: Should this request wait until the operation - has completed before returning, default False - """ - _, data = self.transport.perform_request('POST', _make_path('_snapshot', - repository, snapshot, '_restore'), params=params, body=body) - return data - -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/client/utils.py
Deleted
@@ -1,81 +0,0 @@ -from datetime import date, datetime -from functools import wraps -try: - # PY2 - from urllib import quote_plus -except ImportError: - # PY3 - from urllib.parse import quote_plus - -# parts of URL to be omitted -SKIP_IN_PATH = (None, '', [], ()) - -def _escape(value): - """ - Escape a single value of a URL string or a query parameter. If it is a list - or tuple, turn it into a comma-separated string first. - """ - - # make sequences into comma-separated stings - if isinstance(value, (list, tuple)): - value = u','.join(value) - - # dates and datetimes into isoformat - elif isinstance(value, (date, datetime)): - value = value.isoformat() - - # make bools into true/false strings - elif isinstance(value, bool): - value = str(value).lower() - - # encode strings to utf-8 - if isinstance(value, (type(''), type(u''))): - try: - return value.encode('utf-8') - except UnicodeDecodeError: - # Python 2 and str, no need to re-encode - pass - - return str(value) - -def _make_path(*parts): - """ - Create a URL string from parts, omit all `None` values and empty strings. - Convert lists nad tuples to comma separated values. - """ - #TODO: maybe only allow some parts to be lists/tuples ? - return '/' + '/'.join( - # preserve ',' and '*' in url for nicer URLs in logs - quote_plus(_escape(p), ',*') for p in parts if p not in SKIP_IN_PATH) - -# parameters that apply to all methods -GLOBAL_PARAMS = ('pretty', 'format', ) - -def query_params(*es_query_params): - """ - Decorator that pops all accepted parameters from method's kwargs and puts - them in the params argument. - """ - def _wrapper(func): - @wraps(func) - def _wrapped(*args, **kwargs): - params = kwargs.pop('params', {}) - for p in es_query_params + GLOBAL_PARAMS: - if p in kwargs: - params[p] = _escape(kwargs.pop(p)) - - # don't treat ignore as other params to avoid escaping - if 'ignore' in kwargs: - params['ignore'] = kwargs.pop('ignore') - return func(*args, params=params, **kwargs) - return _wrapped - return _wrapper - - -class NamespacedClient(object): - def __init__(self, client): - self.client = client - - @property - def transport(self): - return self.client.transport
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection
Deleted
-(directory)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/__init__.py
Deleted
@@ -1,5 +0,0 @@ -from .base import Connection -from .http_requests import RequestsHttpConnection -from .http_urllib3 import Urllib3HttpConnection -from .memcached import MemcachedConnection -from .thrift import ThriftConnection, THRIFT_AVAILABLE
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/base.py
Deleted
@@ -1,85 +0,0 @@ -import logging -import json - -from ..exceptions import TransportError, HTTP_EXCEPTIONS - -logger = logging.getLogger('elasticsearch') -tracer = logging.getLogger('elasticsearch.trace') -tracer.propagate = False - -class Connection(object): - """ - Class responsible for maintaining a connection to an Elasticsearch node. It - holds persistent connection pool to it and it's main interface - (`perform_request`) is thread-safe. - - Also responsible for logging. - """ - transport_schema = 'http' - - def __init__(self, host='localhost', port=9200, url_prefix='', timeout=10, **kwargs): - """ - :arg host: hostname of the node (default: localhost) - :arg port: port to use (default: 9200) - :arg url_prefix: optional url prefix for elasticsearch - :arg timeout: default timeout in seconds (default: 10) - """ - self.host = '%s://%s:%s' % (self.transport_schema, host, port) - if url_prefix: - url_prefix = '/' + url_prefix.strip('/') - self.url_prefix = url_prefix - self.timeout = timeout - - def __repr__(self): - return '<%s: %s>' % (self.__class__.__name__, self.host) - - def log_request_success(self, method, full_url, path, body, status_code, response, duration): - """ Log a successful API call. """ - # TODO: optionally pass in params instead of full_url and do urlencode only when needed - def _pretty_json(data): - # pretty JSON in tracer curl logs - try: - return json.dumps(json.loads(data), sort_keys=True, indent=2, separators=(',', ': ')).replace("'", r'\u0027') - except (ValueError, TypeError): - # non-json data or a bulk request - return data - - logger.info( - '%s %s [status:%s request:%.3fs]', method, full_url, - status_code, duration - ) - logger.debug('> %s', body) - logger.debug('< %s', response) - - if tracer.isEnabledFor(logging.INFO): - # include pretty in trace curls - path = path.replace('?', '?pretty&', 1) if '?' in path else path + '?pretty' - if self.url_prefix: - path = path.replace(self.url_prefix, '', 1) - tracer.info("curl -X%s 'http://localhost:9200%s' -d '%s'", method, path, _pretty_json(body) if body else '') - - if tracer.isEnabledFor(logging.DEBUG): - tracer.debug('#[%s] (%.3fs)\n#%s', status_code, duration, _pretty_json(response).replace('\n', '\n#') if response else '') - - def log_request_fail(self, method, full_url, body, duration, status_code=None, exception=None): - """ Log an unsuccessful API call. """ - logger.warning( - '%s %s [status:%s request:%.3fs]', method, full_url, - status_code or 'N/A', duration, exc_info=exception is not None - ) - logger.info('> %s', body) - - def _raise_error(self, status_code, raw_data): - """ Locate appropriate exception and raise it. """ - error_message = raw_data - additional_info = None - try: - additional_info = json.loads(raw_data) - error_message = additional_info.get('error', error_message) - except: - # we don't care what went wrong - pass - - raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) - -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift
Deleted
-(directory)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift/Rest-remote
Deleted
@@ -1,88 +0,0 @@ -#!/usr/bin/env python -# -# Autogenerated by Thrift Compiler (0.9.0) -# -# DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING -# -# options string: py:new_style=true,utf8strings=true -# - -import sys -import pprint -from urlparse import urlparse -from thrift.transport import TTransport -from thrift.transport import TSocket -from thrift.transport import THttpClient -from thrift.protocol import TBinaryProtocol - -import Rest -from ttypes import * - -if len(sys.argv) <= 1 or sys.argv[1] == '--help': - print '' - print 'Usage: ' + sys.argv[0] + ' [-h host[:port]] [-u url] [-f[ramed]] function [arg1 [arg2...]]' - print '' - print 'Functions:' - print ' RestResponse execute(RestRequest request)' - print '' - sys.exit(0) - -pp = pprint.PrettyPrinter(indent = 2) -host = 'localhost' -port = 9090 -uri = '' -framed = False -http = False -argi = 1 - -if sys.argv[argi] == '-h': - parts = sys.argv[argi+1].split(':') - host = parts[0] - if len(parts) > 1: - port = int(parts[1]) - argi += 2 - -if sys.argv[argi] == '-u': - url = urlparse(sys.argv[argi+1]) - parts = url[1].split(':') - host = parts[0] - if len(parts) > 1: - port = int(parts[1]) - else: - port = 80 - uri = url[2] - if url[4]: - uri += '?%s' % url[4] - http = True - argi += 2 - -if sys.argv[argi] == '-f' or sys.argv[argi] == '-framed': - framed = True - argi += 1 - -cmd = sys.argv[argi] -args = sys.argv[argi+1:] - -if http: - transport = THttpClient.THttpClient(host, port, uri) -else: - socket = TSocket.TSocket(host, port) - if framed: - transport = TTransport.TFramedTransport(socket) - else: - transport = TTransport.TBufferedTransport(socket) -protocol = TBinaryProtocol.TBinaryProtocol(transport) -client = Rest.Client(protocol) -transport.open() - -if cmd == 'execute': - if len(args) != 1: - print 'execute requires 1 args' - sys.exit(1) - pp.pprint(client.execute(eval(args[0]),)) - -else: - print 'Unrecognized method %s' % cmd - sys.exit(1) - -transport.close()
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift/Rest.py
Deleted
@@ -1,223 +0,0 @@ -# -# Autogenerated by Thrift Compiler (0.9.0) -# -# DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING -# -# options string: py:new_style=true,utf8strings=true -# - -from thrift.Thrift import TType, TMessageType, TException, TApplicationException -from ttypes import * -from thrift.Thrift import TProcessor -from thrift.transport import TTransport -from thrift.protocol import TBinaryProtocol, TProtocol -try: - from thrift.protocol import fastbinary -except: - fastbinary = None - - -class Iface(object): - def execute(self, request): - """ - Parameters: - - request - """ - pass - - -class Client(Iface): - def __init__(self, iprot, oprot=None): - self._iprot = self._oprot = iprot - if oprot is not None: - self._oprot = oprot - self._seqid = 0 - - def execute(self, request): - """ - Parameters: - - request - """ - self.send_execute(request) - return self.recv_execute() - - def send_execute(self, request): - self._oprot.writeMessageBegin('execute', TMessageType.CALL, self._seqid) - args = execute_args() - args.request = request - args.write(self._oprot) - self._oprot.writeMessageEnd() - self._oprot.trans.flush() - - def recv_execute(self, ): - (fname, mtype, rseqid) = self._iprot.readMessageBegin() - if mtype == TMessageType.EXCEPTION: - x = TApplicationException() - x.read(self._iprot) - self._iprot.readMessageEnd() - raise x - result = execute_result() - result.read(self._iprot) - self._iprot.readMessageEnd() - if result.success is not None: - return result.success - raise TApplicationException(TApplicationException.MISSING_RESULT, "execute failed: unknown result"); - - -class Processor(Iface, TProcessor): - def __init__(self, handler): - self._handler = handler - self._processMap = {} - self._processMap["execute"] = Processor.process_execute - - def process(self, iprot, oprot): - (name, type, seqid) = iprot.readMessageBegin() - if name not in self._processMap: - iprot.skip(TType.STRUCT) - iprot.readMessageEnd() - x = TApplicationException(TApplicationException.UNKNOWN_METHOD, 'Unknown function %s' % (name)) - oprot.writeMessageBegin(name, TMessageType.EXCEPTION, seqid) - x.write(oprot) - oprot.writeMessageEnd() - oprot.trans.flush() - return - else: - self._processMap[name](self, seqid, iprot, oprot) - return True - - def process_execute(self, seqid, iprot, oprot): - args = execute_args() - args.read(iprot) - iprot.readMessageEnd() - result = execute_result() - result.success = self._handler.execute(args.request) - oprot.writeMessageBegin("execute", TMessageType.REPLY, seqid) - result.write(oprot) - oprot.writeMessageEnd() - oprot.trans.flush() - - -# HELPER FUNCTIONS AND STRUCTURES - -class execute_args(object): - """ - Attributes: - - request - """ - - thrift_spec = ( - None, # 0 - (1, TType.STRUCT, 'request', (RestRequest, RestRequest.thrift_spec), None, ), # 1 - ) - - def __init__(self, request=None,): - self.request = request - - def read(self, iprot): - if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None and fastbinary is not None: - fastbinary.decode_binary(self, iprot.trans, (self.__class__, self.thrift_spec)) - return - iprot.readStructBegin() - while True: - (fname, ftype, fid) = iprot.readFieldBegin() - if ftype == TType.STOP: - break - if fid == 1: - if ftype == TType.STRUCT: - self.request = RestRequest() - self.request.read(iprot) - else: - iprot.skip(ftype) - else: - iprot.skip(ftype) - iprot.readFieldEnd() - iprot.readStructEnd() - - def write(self, oprot): - if oprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and self.thrift_spec is not None and fastbinary is not None: - oprot.trans.write(fastbinary.encode_binary(self, (self.__class__, self.thrift_spec))) - return - oprot.writeStructBegin('execute_args') - if self.request is not None: - oprot.writeFieldBegin('request', TType.STRUCT, 1) - self.request.write(oprot) - oprot.writeFieldEnd() - oprot.writeFieldStop() - oprot.writeStructEnd() - - def validate(self): - if self.request is None: - raise TProtocol.TProtocolException(message='Required field request is unset!') - return - - - def __repr__(self): - L = ['%s=%r' % (key, value) - for key, value in self.__dict__.iteritems()] - return '%s(%s)' % (self.__class__.__name__, ', '.join(L)) - - def __eq__(self, other): - return isinstance(other, self.__class__) and self.__dict__ == other.__dict__ - - def __ne__(self, other): - return not (self == other) - -class execute_result(object): - """ - Attributes: - - success - """ - - thrift_spec = ( - (0, TType.STRUCT, 'success', (RestResponse, RestResponse.thrift_spec), None, ), # 0 - ) - - def __init__(self, success=None,): - self.success = success - - def read(self, iprot): - if iprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and isinstance(iprot.trans, TTransport.CReadableTransport) and self.thrift_spec is not None and fastbinary is not None: - fastbinary.decode_binary(self, iprot.trans, (self.__class__, self.thrift_spec)) - return - iprot.readStructBegin() - while True: - (fname, ftype, fid) = iprot.readFieldBegin() - if ftype == TType.STOP: - break - if fid == 0: - if ftype == TType.STRUCT: - self.success = RestResponse() - self.success.read(iprot) - else: - iprot.skip(ftype) - else: - iprot.skip(ftype) - iprot.readFieldEnd() - iprot.readStructEnd() - - def write(self, oprot): - if oprot.__class__ == TBinaryProtocol.TBinaryProtocolAccelerated and self.thrift_spec is not None and fastbinary is not None:
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift/__init__.py
Deleted
@@ -1,1 +0,0 @@ -__all__ = ['ttypes', 'constants', 'Rest']
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift/constants.py
Deleted
@@ -1,11 +0,0 @@ -# -# Autogenerated by Thrift Compiler (0.9.0) -# -# DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING -# -# options string: py:new_style=true,utf8strings=true -# - -from thrift.Thrift import TType, TMessageType, TException, TApplicationException -from ttypes import * -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/esthrift/ttypes.py
Deleted
@@ -1,412 +0,0 @@ -# -# Autogenerated by Thrift Compiler (0.9.0) -# -# DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING -# -# options string: py:new_style=true,utf8strings=true -# - -from thrift.Thrift import TType, TMessageType, TException, TApplicationException - -from thrift.transport import TTransport -from thrift.protocol import TBinaryProtocol, TProtocol -try: - from thrift.protocol import fastbinary -except: - fastbinary = None - - -class Method(object): - GET = 0 - PUT = 1 - POST = 2 - DELETE = 3 - HEAD = 4 - OPTIONS = 5 - - _VALUES_TO_NAMES = { - 0: "GET", - 1: "PUT", - 2: "POST", - 3: "DELETE", - 4: "HEAD", - 5: "OPTIONS", - } - - _NAMES_TO_VALUES = { - "GET": 0, - "PUT": 1, - "POST": 2, - "DELETE": 3, - "HEAD": 4, - "OPTIONS": 5, - } - -class Status(object): - CONT = 100 - SWITCHING_PROTOCOLS = 101 - OK = 200 - CREATED = 201 - ACCEPTED = 202 - NON_AUTHORITATIVE_INFORMATION = 203 - NO_CONTENT = 204 - RESET_CONTENT = 205 - PARTIAL_CONTENT = 206 - MULTI_STATUS = 207 - MULTIPLE_CHOICES = 300 - MOVED_PERMANENTLY = 301 - FOUND = 302 - SEE_OTHER = 303 - NOT_MODIFIED = 304 - USE_PROXY = 305 - TEMPORARY_REDIRECT = 307 - BAD_REQUEST = 400 - UNAUTHORIZED = 401 - PAYMENT_REQUIRED = 402 - FORBIDDEN = 403 - NOT_FOUND = 404 - METHOD_NOT_ALLOWED = 405 - NOT_ACCEPTABLE = 406 - PROXY_AUTHENTICATION = 407 - REQUEST_TIMEOUT = 408 - CONFLICT = 409 - GONE = 410 - LENGTH_REQUIRED = 411 - PRECONDITION_FAILED = 412 - REQUEST_ENTITY_TOO_LARGE = 413 - REQUEST_URI_TOO_LONG = 414 - UNSUPPORTED_MEDIA_TYPE = 415 - REQUESTED_RANGE_NOT_SATISFIED = 416 - EXPECTATION_FAILED = 417 - UNPROCESSABLE_ENTITY = 422 - LOCKED = 423 - FAILED_DEPENDENCY = 424 - INTERNAL_SERVER_ERROR = 500 - NOT_IMPLEMENTED = 501 - BAD_GATEWAY = 502 - SERVICE_UNAVAILABLE = 503 - GATEWAY_TIMEOUT = 504 - INSUFFICIENT_STORAGE = 506 - - _VALUES_TO_NAMES = { - 100: "CONT", - 101: "SWITCHING_PROTOCOLS", - 200: "OK", - 201: "CREATED", - 202: "ACCEPTED", - 203: "NON_AUTHORITATIVE_INFORMATION", - 204: "NO_CONTENT", - 205: "RESET_CONTENT", - 206: "PARTIAL_CONTENT", - 207: "MULTI_STATUS", - 300: "MULTIPLE_CHOICES", - 301: "MOVED_PERMANENTLY", - 302: "FOUND", - 303: "SEE_OTHER", - 304: "NOT_MODIFIED", - 305: "USE_PROXY", - 307: "TEMPORARY_REDIRECT", - 400: "BAD_REQUEST", - 401: "UNAUTHORIZED", - 402: "PAYMENT_REQUIRED", - 403: "FORBIDDEN", - 404: "NOT_FOUND", - 405: "METHOD_NOT_ALLOWED", - 406: "NOT_ACCEPTABLE", - 407: "PROXY_AUTHENTICATION", - 408: "REQUEST_TIMEOUT", - 409: "CONFLICT", - 410: "GONE", - 411: "LENGTH_REQUIRED", - 412: "PRECONDITION_FAILED", - 413: "REQUEST_ENTITY_TOO_LARGE", - 414: "REQUEST_URI_TOO_LONG", - 415: "UNSUPPORTED_MEDIA_TYPE", - 416: "REQUESTED_RANGE_NOT_SATISFIED", - 417: "EXPECTATION_FAILED", - 422: "UNPROCESSABLE_ENTITY", - 423: "LOCKED", - 424: "FAILED_DEPENDENCY", - 500: "INTERNAL_SERVER_ERROR", - 501: "NOT_IMPLEMENTED", - 502: "BAD_GATEWAY", - 503: "SERVICE_UNAVAILABLE", - 504: "GATEWAY_TIMEOUT", - 506: "INSUFFICIENT_STORAGE", - } - - _NAMES_TO_VALUES = { - "CONT": 100, - "SWITCHING_PROTOCOLS": 101, - "OK": 200, - "CREATED": 201, - "ACCEPTED": 202, - "NON_AUTHORITATIVE_INFORMATION": 203, - "NO_CONTENT": 204, - "RESET_CONTENT": 205, - "PARTIAL_CONTENT": 206, - "MULTI_STATUS": 207, - "MULTIPLE_CHOICES": 300, - "MOVED_PERMANENTLY": 301, - "FOUND": 302, - "SEE_OTHER": 303, - "NOT_MODIFIED": 304, - "USE_PROXY": 305, - "TEMPORARY_REDIRECT": 307, - "BAD_REQUEST": 400, - "UNAUTHORIZED": 401, - "PAYMENT_REQUIRED": 402, - "FORBIDDEN": 403, - "NOT_FOUND": 404, - "METHOD_NOT_ALLOWED": 405, - "NOT_ACCEPTABLE": 406, - "PROXY_AUTHENTICATION": 407, - "REQUEST_TIMEOUT": 408, - "CONFLICT": 409, - "GONE": 410, - "LENGTH_REQUIRED": 411, - "PRECONDITION_FAILED": 412, - "REQUEST_ENTITY_TOO_LARGE": 413, - "REQUEST_URI_TOO_LONG": 414, - "UNSUPPORTED_MEDIA_TYPE": 415, - "REQUESTED_RANGE_NOT_SATISFIED": 416, - "EXPECTATION_FAILED": 417, - "UNPROCESSABLE_ENTITY": 422, - "LOCKED": 423, - "FAILED_DEPENDENCY": 424, - "INTERNAL_SERVER_ERROR": 500, - "NOT_IMPLEMENTED": 501, - "BAD_GATEWAY": 502, - "SERVICE_UNAVAILABLE": 503, - "GATEWAY_TIMEOUT": 504, - "INSUFFICIENT_STORAGE": 506, - } - - -class RestRequest(object): - """ - Attributes: - - method - - uri - - parameters - - headers - - body - """ - - thrift_spec = ( - None, # 0 - (1, TType.I32, 'method', None, None, ), # 1 - (2, TType.STRING, 'uri', None, None, ), # 2
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/http_requests.py
Deleted
@@ -1,61 +0,0 @@ -import time -try: - import requests - REQUESTS_AVAILABLE = True -except ImportError: - REQUESTS_AVAILABLE = False -try: - from urllib import urlencode -except ImportError: - from urllib.parse import urlencode - -from .base import Connection -from ..exceptions import ConnectionError, ImproperlyConfigured - -class RequestsHttpConnection(Connection): - """ - Connection using the `requests` library. - - :arg http_auth: optional http auth information as either ':' separated - string or a tuple - :arg use_ssl: use ssl for the connection if `True` - """ - def __init__(self, host='localhost', port=9200, http_auth=None, use_ssl=False, **kwargs): - if not REQUESTS_AVAILABLE: - raise ImproperlyConfigured("Please install requests to use RequestsHttpConnection.") - - super(RequestsHttpConnection, self).__init__(host= host, port=port, **kwargs) - self.session = requests.session() - if http_auth is not None: - if not isinstance(http_auth, (tuple, list)): - http_auth = http_auth.split(':', 1) - http_auth = tuple(http_auth) - self.session.auth = http_auth - self.base_url = 'http%s://%s:%d%s' % ( - 's' if use_ssl else '', - host, port, self.url_prefix - ) - - - def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): - url = self.base_url + url - if params: - url = '%s?%s' % (url, urlencode(params or {})) - - start = time.time() - try: - response = self.session.request(method, url, data=body, timeout=timeout or self.timeout) - duration = time.time() - start - raw_data = response.text - except (requests.ConnectionError, requests.Timeout) as e: - self.log_request_fail(method, url, body, time.time() - start, exception=e) - raise ConnectionError('N/A', str(e), e) - - # raise errors based on http status codes, let the client handle those if needed - if not (200 <= response.status_code < 300) and response.status_code not in ignore: - self.log_request_fail(method, url, body, duration, response.status_code) - self._raise_error(response.status_code, raw_data) - - self.log_request_success(method, url, response.request.path_url, body, response.status_code, raw_data, duration) - - return response.status_code, response.headers, raw_data
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/http_urllib3.py
Deleted
@@ -1,61 +0,0 @@ -import time -import urllib3 -try: - from urllib import urlencode -except ImportError: - from urllib.parse import urlencode - -from .base import Connection -from ..exceptions import ConnectionError - -class Urllib3HttpConnection(Connection): - """ - Default connection class using the `urllib3` library and the http protocol. - - :arg http_auth: optional http auth information as either ':' separated - string or a tuple - :arg use_ssl: use ssl for the connection if `True` - :arg maxsize: the maximum number of connections which will be kept open to - this host. - """ - def __init__(self, host='localhost', port=9200, http_auth=None, use_ssl=False, maxsize=10, **kwargs): - super(Urllib3HttpConnection, self).__init__(host=host, port=port, **kwargs) - headers = {} - if http_auth is not None: - if isinstance(http_auth, (tuple, list)): - http_auth = ':'.join(http_auth) - headers = urllib3.make_headers(basic_auth=http_auth) - - pool_class = urllib3.HTTPConnectionPool - if use_ssl: - pool_class = urllib3.HTTPSConnectionPool - - self.pool = pool_class(host, port=port, timeout=self.timeout, headers=headers, maxsize=maxsize) - - def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): - url = self.url_prefix + url - if params: - url = '%s?%s' % (url, urlencode(params or {})) - full_url = self.host + url - - start = time.time() - try: - kw = {} - if timeout: - kw['timeout'] = timeout - response = self.pool.urlopen(method, url, body, **kw) - duration = time.time() - start - raw_data = response.data.decode('utf-8') - except Exception as e: - self.log_request_fail(method, full_url, body, time.time() - start, exception=e) - raise ConnectionError('N/A', str(e), e) - - if not (200 <= response.status < 300) and response.status not in ignore: - self.log_request_fail(method, url, body, duration, response.status) - self._raise_error(response.status, raw_data) - - self.log_request_success(method, full_url, url, body, response.status, - raw_data, duration) - - return response.status, response.getheaders(), raw_data -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/memcached.py
Deleted
@@ -1,83 +0,0 @@ -import time -import json -try: - from urllib import urlencode -except ImportError: - from urllib.parse import urlencode - -from ..exceptions import TransportError, ConnectionError, ImproperlyConfigured -from .pooling import PoolingConnection - -class MemcachedConnection(PoolingConnection): - """ - Client using the `pylibmc` python library to communicate with elasticsearch - using the memcached protocol. Requires plugin in the cluster. - - See https://github.com/elasticsearch/elasticsearch-transport-memcached for more details. - """ - transport_schema = 'memcached' - - method_map = { - 'PUT': 'set', - 'POST': 'set', - 'DELETE': 'delete', - 'HEAD': 'get', - 'GET': 'get', - } - - def __init__(self, host='localhost', port=11211, **kwargs): - try: - import pylibmc - except ImportError: - raise ImproperlyConfigured("You need to install pylibmc to use the MemcachedConnection class.") - super(MemcachedConnection, self).__init__(host=host, port=port, **kwargs) - self._make_connection = lambda: pylibmc.Client(['%s:%s' % (host, port)], behaviors={"tcp_nodelay": True}) - - def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): - mc = self._get_connection() - url = self.url_prefix + url - if params: - url = '%s?%s' % (url, urlencode(params or {})) - full_url = self.host + url - - mc_method = self.method_map.get(method, 'get') - - start = time.time() - try: - status = 200 - if mc_method == 'set': - # no response from set commands - response = '' - if not json.dumps(mc.set(url, body)): - status = 500 - else: - response = mc.get(url) - - duration = time.time() - start - if response: - response = response.decode('utf-8') - except Exception as e: - self.log_request_fail(method, full_url, body, time.time() - start, exception=e) - raise ConnectionError('N/A', str(e), e) - finally: - self._release_connection(mc) - - # try not to load the json every time - if response and response[0] == '{' and ('"status"' in response or '"error"' in response): - data = json.loads(response) - if 'status' in data and isinstance(data['status'], int): - status = data['status'] - elif 'error' in data: - raise TransportError('N/A', data['error']) - - if not (200 <= status < 300) and status not in ignore: - self.log_request_fail(method, url, body, duration, status) - self._raise_error(status, response) - - self.log_request_success(method, full_url, url, body, status, - response, duration) - - return status, {}, response - - -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/pooling.py
Deleted
@@ -1,22 +0,0 @@ -from .base import Connection - -class PoolingConnection(Connection): - def __init__(self, *args, **kwargs): - self._max_pool_size = kwargs.pop('max_connection_pool_size', 50) - self._free_connections = [] - self._in_use_connections = set() - super(PoolingConnection, self).__init__(*args, **kwargs) - - def _get_connection(self): - try: - con = self._free_connections.pop() - except IndexError: - con = self._make_connection() - - self._in_use_connections.add(con) - return con - - def _release_connection(self, con): - self._in_use_connections.remove(con) - self._free_connections.append(con) -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection/thrift.py
Deleted
@@ -1,82 +0,0 @@ -from __future__ import absolute_import -import time - -try: - from .esthrift import Rest - from .esthrift.ttypes import Method, RestRequest - - from thrift.transport import TTransport, TSocket, TSSLSocket - from thrift.protocol import TBinaryProtocol - from thrift.Thrift import TException - THRIFT_AVAILABLE = True -except ImportError: - THRIFT_AVAILABLE = False - -from ..exceptions import ConnectionError, ImproperlyConfigured -from .pooling import PoolingConnection - -class ThriftConnection(PoolingConnection): - """ - Connection using the `thrift` protocol to communicate with elasticsearch. - - See https://github.com/elasticsearch/elasticsearch-transport-thrift for additional info. - """ - transport_schema = 'thrift' - - def __init__(self, host='localhost', port=9500, framed_transport=False, use_ssl=False, **kwargs): - """ - :arg framed_transport: use `TTransport.TFramedTransport` instead of - `TTransport.TBufferedTransport` - """ - if not THRIFT_AVAILABLE: - raise ImproperlyConfigured("Thrift is not available.") - - super(ThriftConnection, self).__init__(host=host, port=port, **kwargs) - self._framed_transport = framed_transport - self._tsocket_class = TSocket.TSocket - if use_ssl: - self._tsocket_class = TSSLSocket.TSSLSocket - self._tsocket_args = (host, port) - - def _make_connection(self): - socket = self._tsocket_class(*self._tsocket_args) - socket.setTimeout(self.timeout * 1000.0) - if self._framed_transport: - transport = TTransport.TFramedTransport(socket) - else: - transport = TTransport.TBufferedTransport(socket) - - protocol = TBinaryProtocol.TBinaryProtocolAccelerated(transport) - client = Rest.Client(protocol) - transport.open() - return client - - def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): - request = RestRequest(method=Method._NAMES_TO_VALUES[method.upper()], uri=url, - parameters=params, body=body) - - start = time.time() - tclient = None - try: - tclient = self._get_connection() - response = tclient.execute(request) - duration = time.time() - start - except TException as e: - self.log_request_fail(method, url, body, time.time() - start, exception=e) - raise ConnectionError('N/A', str(e), e) - finally: - if tclient: - self._release_connection(tclient) - - if not (200 <= response.status < 300) and response.status not in ignore: - self.log_request_fail(method, url, body, duration, response.status) - self._raise_error(response.status, response.body) - - self.log_request_success(method, url, url, body, response.status, - response.body, duration) - - headers = {} - if response.headers: - headers = dict((k.lower(), v) for k, v in response.headers.items()) - return response.status, headers, response.body or '' -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/connection_pool.py
Deleted
@@ -1,208 +0,0 @@ -import time -import random -import logging - -try: - from Queue import PriorityQueue, Empty -except ImportError: - from queue import PriorityQueue, Empty - -logger = logging.getLogger('elasticsearch') - -class ConnectionSelector(object): - """ - Simple class used to select a connection from a list of currently live - connection instances. In init time it is passed a dictionary containing all - the connections' options which it can then use during the selection - process. When the `select` method is called it is given a list of - *currently* live connections to choose from. - - The options dictionary is the one that has been passed to - :class:`~elasticsearch.Transport` as `hosts` param and the same that is - used to construct the Connection object itself. When the Connection was - created from information retrieved from the cluster via the sniffing - process it will be the dictionary returned by the `host_info_callback`. - - Example of where this would be useful is a zone-aware selector that would - only select connections from it's own zones and only fall back to other - connections where there would be none in it's zones. - """ - def __init__(self, opts): - """ - :arg opts: dictionary of connection instances and their options - """ - self.connection_opts = opts - - def select(self, connections): - """ - Select a connection from the given list. - - :arg connections: list of live connections to choose from - """ - pass - - -class RandomSelector(ConnectionSelector): - """ - Select a connection at random - """ - def select(self, connections): - return random.choice(connections) - - -class RoundRobinSelector(ConnectionSelector): - """ - Selector using round-robin. - """ - def __init__(self, opts): - super(RoundRobinSelector, self).__init__(opts) - self.rr = -1 - - def select(self, connections): - self.rr += 1 - self.rr %= len(connections) - return connections[self.rr] - - -class ConnectionPool(object): - """ - Container holding the :class:`~elasticsearch.Connection` instances, - managing the selection process (via a - :class:`~elasticsearch.ConnectionSelector`) and dead connections. - - It's only interactions are with the :class:`~elasticsearch.Transport` class - that drives all the actions within `ConnectionPool`. - - Initially connections are stored on the class as a list and, along with the - connection options, get passed to the `ConnectionSelector` instance for - future reference. - - Upon each request the `Transport` will ask for a `Connection` via the - `get_connection` method. If the connection fails (it's `perform_request` - raises a `ConnectionError`) it will be marked as dead (via `mark_dead`) and - put on a timeout (if it fails N times in a row the timeout is exponentially - longer - the formula is `default_timeout * 2 ** (fail_count - 1)`). When - the timeout is over the connection will be resurrected and returned to the - live pool. A connection that has been peviously marked as dead and - succeedes will be marked as live (it's fail count will be deleted). - """ - def __init__(self, connections, dead_timeout=60, timeout_cutoff=5, - selector_class=RoundRobinSelector, randomize_hosts=True, **kwargs): - """ - :arg connections: list of tuples containing the - :class:`~elasticsearch.Connection` instance and it's options - :arg dead_timeout: number of seconds a connection should be retired for - after a failure, increases on consecutive failures - :arg timeout_cutoff: number of consecutive failures after which the - timeout doesn't increase - :arg selector_class: :class:`~elasticsearch.ConnectionSelector` - subclass to use - :arg randomize_hosts: shuffle the list of connections upon arrival to - avoid dog piling effect across processes - """ - self.connection_opts = connections - self.connections = [c for (c, opts) in connections] - # PriorityQueue for thread safety and ease of timeout management - self.dead = PriorityQueue(len(self.connections)) - self.dead_count = {} - - if randomize_hosts: - # randomize the connection list to avoid all clients hitting same node - # after startup/restart - random.shuffle(self.connections) - - # default timeout after which to try resurrecting a connection - self.dead_timeout = dead_timeout - self.timeout_cutoff = timeout_cutoff - - self.selector = selector_class(dict(connections)) - - def mark_dead(self, connection, now=None): - """ - Mark the connection as dead (failed). Remove it from the live pool and - put it on a timeout. - - :arg connection: the failed instance - """ - # allow inject for testing purposes - now = now if now else time.time() - try: - self.connections.remove(connection) - except ValueError: - # connection not alive or another thread marked it already, ignore - return - else: - dead_count = self.dead_count.get(connection, 0) + 1 - self.dead_count[connection] = dead_count - timeout = self.dead_timeout * 2 ** min(dead_count - 1, self.timeout_cutoff) - self.dead.put((now + timeout, connection)) - logger.warning( - 'Connection %r has failed for %i times in a row, putting on %i second timeout.', - connection, dead_count, timeout - ) - - def mark_live(self, connection): - """ - Mark connection as healthy after a resurrection. Resets the fail - counter for the connection. - - :arg connection: the connection to redeem - """ - try: - del self.dead_count[connection] - except KeyError: - # race condition, safe to ignore - pass - - def resurrect(self, force=False): - """ - Attempt to resurrect a connection from the dead pool. It will try to - locate one (not all) eligible (it's timeout is over) connection to - return to th live pool. - - :arg force: resurrect a connection even if there is none eligible (used - when we have no live connections) - - """ - # no dead connections - if self.dead.empty(): - return - - try: - # retrieve a connection to check - timeout, connection = self.dead.get(block=False) - except Empty: - # other thread has been faster and the queue is now empty - return - - if not force and timeout > time.time(): - # return it back if not eligible and not forced - self.dead.put((timeout, connection)) - return - - # either we were forced or the connection is elligible to be retried - self.connections.append(connection) - logger.info('Resurrecting connection %r (force=%s).', connection, force) - - def get_connection(self): - """ - Return a connection from the pool using the `ConnectionSelector` - instance. - - It tries to resurrect eligible connections, forces a resurrection when - no connections are availible and passes the list of live connections to - the selector instance to choose from. - - Returns a connection instance and it's current fail count. - """ - self.resurrect() -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/exceptions.py
Deleted
@@ -1,61 +0,0 @@ -__all__ = [ - 'ImproperlyConfigured', 'ElasticsearchException', 'SerializationError', - 'TransportError', 'NotFoundError', 'ConflictError', 'RequestError' -] - -class ImproperlyConfigured(Exception): - pass - -class ElasticsearchException(Exception): - pass - - -class SerializationError(ElasticsearchException): - pass - - -class TransportError(ElasticsearchException): - """ Exception raised when ES returns a non-OK (>=400) HTTP status code. """ - @property - def status_code(self): - """ The HTTP status code of the response that precipitated the error. """ - return self.args[0] - - @property - def error(self): - """ A string error message. """ - return self.args[1] - - @property - def info(self): - """ Dict of returned error info from ES, where applicable. """ - return self.args[2] - - def __str__(self): - return 'TransportError(%s, %r)' % (self.status_code, self.error) - - -class ConnectionError(TransportError): - """ Error raised when there was an exception while talking to ES. """ - def __str__(self): - return 'ConnectionError(%s) caused by: %s(%s)' % ( - self.error, self.info.__class__.__name__, self.info) - - -class NotFoundError(TransportError): - pass - - -class ConflictError(TransportError): - pass - - -class RequestError(TransportError): - pass - -# more generic mappings from status_code to python exceptions -HTTP_EXCEPTIONS = { - 400: RequestError, - 404: NotFoundError, - 409: ConflictError, -}
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/helpers.py
Deleted
@@ -1,217 +0,0 @@ -from itertools import islice -from operator import methodcaller -try: - from itertools import imap as map -except ImportError: - pass # python 3, use builtin map - -from elasticsearch.exceptions import ElasticsearchException - -class BulkIndexError(ElasticsearchException): - @property - def errors(self): - """ List of errors from execution of the last chunk. """ - return self.args[1] - - -def expand_action(data): - """ - From one document or action definition passed in by the user extract the - action/data lines needed for elasticsearch's - :meth:`~elasticsearch.Elasticsearch.bulk` api. - """ - # make sure we don't alter the action - data = data.copy() - op_type = data.pop('_op_type', 'index') - action = {op_type: {}} - for key in ('_index', '_parent', '_percolate', '_routing', '_timestamp', - '_ttl', '_type', '_version', '_id', '_retry_on_conflict'): - if key in data: - action[op_type][key] = data.pop(key) - - # no data payload for delete - if op_type == 'delete': - return action, None - - return action, data.get('_source', data) - - -def streaming_bulk(client, actions, chunk_size=500, raise_on_error=False, expand_action_callback=expand_action, **kwargs): - """ - Streaming bulk consumes actions from the iterable passed in and yields - results per action. For non-streaming usecases use - :func:`~elasticsearch.helpers.bulk` which is a wrapper around streaming - bulk that returns summary information about the bulk operation once the - entire input is consumed and sent. - - This function expects the action to be in the format as returned by - :meth:`~elasticsearch.Elasticsearch.search`, for example:: - - { - '_index': 'index-name', - '_type': 'document', - '_id': 42, - '_parent': 5, - '_ttl': '1d', - '_source': { - ... - } - } - - Alternatively, if `_source` is not present, it will pop all metadata fields - from the doc and use the rest as the document data. - - Alternative actions (`_op_type` field defaults to `index`) can be sent as - well:: - - { - '_op_type': 'delete', - '_index': 'index-name', - '_type': 'document', - '_id': 42, - } - { - '_op_type': 'update', - '_index': 'index-name', - '_type': 'document', - '_id': 42, - 'doc': {'question': 'The life, universe and everything.'} - } - - :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use - :arg actions: iterable containing the actions to be executed - :arg chunk_size: number of docs in one chunk sent to es (default: 500) - :arg raise_on_error: raise `BulkIndexError` containing errors (as `.errors` - from the execution of the last chunk) - :arg expand_action_callback: callback executed on each action passed in, - should return a tuple containing the action line and the data line - (`None` if data line should be omitted). - """ - actions = map(expand_action_callback, actions) - - # if raise on error is set, we need to collect errors per chunk before raising them - errors = [] - - while True: - chunk = islice(actions, chunk_size) - bulk_actions = [] - for action, data in chunk: - bulk_actions.append(action) - if data is not None: - bulk_actions.append(data) - - if not bulk_actions: - return - - # send the actual request - resp = client.bulk(bulk_actions, **kwargs) - - # go through request-reponse pairs and detect failures - for op_type, item in map(methodcaller('popitem'), resp['items']): - ok = 200 <= item.get('status', 500) < 300 - if not ok and raise_on_error: - errors.append({op_type: item}) - - if not errors: - # if we are not just recording all errors to be able to raise - # them all at once, yield items individually - yield ok, {op_type: item} - - if errors: - raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors) - -def bulk(client, actions, stats_only=False, **kwargs): - """ - Helper for the :meth:`~elasticsearch.Elasticsearch.bulk` api that provides - a more human friendly interface - it consumes an iterator of actions and - sends them to elasticsearch in chunks. It returns a tuple with summary - information - number of successfully executed actions and either list of - errors or number of errors if `stats_only` is set to `True`. - - See :func:`~elasticsearch.helpers.streaming_bulk` for more information - and accepted formats. - - :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use - :arg actions: iterator containing the actions - :arg stats_only: if `True` only report number of successful/failed - operations instead of just number of successful and a list of error responses - - Any additional keyword arguments will be passed to - :func:`~elasticsearch.helpers.streaming_bulk` which is used to execute - the operation. - """ - success, failed = 0, 0 - - # list of errors to be collected is not stats_only - errors = [] - - for ok, item in streaming_bulk(client, actions, **kwargs): - # go through request-reponse pairs and detect failures - if not ok: - if not stats_only: - errors.append(item) - failed += 1 - else: - success += 1 - - return success, failed if stats_only else errors - -# preserve the name for backwards compatibility -bulk_index = bulk - -def scan(client, query=None, scroll='5m', **kwargs): - """ - Simple abstraction on top of the - :meth:`~elasticsearch.Elasticsearch.scroll` api - a simple iterator that - yields all hits as returned by underlining scroll requests. - - :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use - :arg query: body for the :meth:`~elasticsearch.Elasticsearch.search` api - :arg scroll: Specify how long a consistent view of the index should be - maintained for scrolled search - - Any additional keyword arguments will be passed to the initial - :meth:`~elasticsearch.Elasticsearch.search` call. - """ - # initial search to - resp = client.search(body=query, search_type='scan', scroll=scroll, **kwargs) - - scroll_id = resp['_scroll_id'] - - while True: - resp = client.scroll(scroll_id, scroll=scroll) - if not resp['hits']['hits']: - break - for hit in resp['hits']['hits']: - yield hit - scroll_id = resp['_scroll_id'] - -def reindex(client, source_index, target_index, target_client=None, chunk_size=500, scroll='5m'): - """ - Reindex all documents from one index to another, potentially (if - `target_client` is specified) on a different cluster. - - .. note:: - - This helper doesn't transfer mappings, just the data. - - :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use (for - read if `target_client` is specified as well)
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/serializer.py
Deleted
@@ -1,70 +0,0 @@ -import json -from datetime import date, datetime -from decimal import Decimal - -from .exceptions import SerializationError, ImproperlyConfigured - -class TextSerializer(object): - mimetype = 'text/plain' - - def loads(self, s): - return s - - def dumps(self, data): - if isinstance(data, (type(''), type(u''))): - return data - - raise SerializationError('Cannot serialize %r into text.' % data) - -class JSONSerializer(object): - mimetype = 'application/json' - - def default(self, data): - if isinstance(data, (date, datetime)): - return data.isoformat() - elif isinstance(data, Decimal): - return float(data) - raise TypeError - - def loads(self, s): - try: - return json.loads(s) - except (ValueError, TypeError) as e: - raise SerializationError(s, e) - - def dumps(self, data): - # don't serialize strings - if isinstance(data, (type(''), type(u''))): - return data - - try: - return json.dumps(data, default=self.default) - except (ValueError, TypeError) as e: - raise SerializationError(data, e) - -DEFAULT_SERIALIZERS = { - JSONSerializer.mimetype: JSONSerializer(), - TextSerializer.mimetype: TextSerializer(), -} - -class Deserializer(object): - def __init__(self, serializers, default_mimetype='application/json'): - try: - self.default = serializers[default_mimetype] - except KeyError: - raise ImproperlyConfigured('Cannot find default serializer (%s)' % default_mimetype) - self.serializers = serializers - - def loads(self, s, mimetype=None): - if not mimetype: - deserializer = self.default - else: - # split out charset - mimetype = mimetype.split(';', 1)[0] - try: - deserializer = self.serializers[mimetype] - except KeyError: - raise SerializationError('Unknown mimetype, unable to deserialize: %s' % mimetype) - - return deserializer.loads(s) -
View file
elasticsearch-1.0.0.tar.gz/elasticsearch/transport.py
Deleted
@@ -1,287 +0,0 @@ -import re -import time - -from .connection import Urllib3HttpConnection -from .connection_pool import ConnectionPool -from .serializer import JSONSerializer, Deserializer, DEFAULT_SERIALIZERS -from .exceptions import ConnectionError, TransportError, SerializationError - -# get ip/port from "inet[wind/127.0.0.1:9200]" -ADDRESS_RE = re.compile(r'/(?P<host>[\.:0-9a-f]*):(?P<port>[0-9]+)\]?$') - - -def get_host_info(node_info, host): - """ - Simple callback that takes the node info from `/_cluster/nodes` and a - parsed connection information and return the connection information. If - `None` is returned this node will be skipped. - - Useful for filtering nodes (by proximity for example) or if additional - information needs to be provided for the :class:`~elasticsearch.Connection` - class. - - :arg node_info: node information from `/_cluster/nodes` - :arg host: connection information (host, port) extracted from the node info - """ - return host - -class Transport(object): - """ - Encapsulation of transport-related to logic. Handles instantiation of the - individual connections as well as creating a connection pool to hold them. - - Main interface is the `perform_request` method. - """ - def __init__(self, hosts, connection_class=Urllib3HttpConnection, - connection_pool_class=ConnectionPool, host_info_callback=get_host_info, - sniff_on_start=False, sniffer_timeout=None, sniff_timeout=.1, - sniff_on_connection_fail=False, serializer=JSONSerializer(), serializers=None, - default_mimetype='application/json', max_retries=3, - send_get_body_as='GET', **kwargs): - """ - :arg hosts: list of dictionaries, each containing keyword arguments to - create a `connection_class` instance - :arg connection_class: subclass of :class:`~elasticsearch.Connection` to use - :arg connection_pool_class: subclass of :class:`~elasticsearch.ConnectionPool` to use - :arg host_info_callback: callback responsible for taking the node information from - `/_cluser/nodes`, along with already extracted information, and - producing a list of arguments (same as `hosts` parameter) - :arg sniff_on_start: flag indicating whether to obtain a list of nodes - from the cluser at startup time - :arg sniffer_timeout: number of seconds between automatic sniffs - :arg sniff_on_connection_fail: flag controlling if connection failure triggers a sniff - :arg sniff_timeout: timeout used for the sniff request - it should be a - fast api call and we are talking potentially to more nodes so we want - to fail quickly. - :arg serializer: serializer instance - :arg serializers: optional dict of serializer instances that will be - used for deserializing data coming from the server. (key is the mimetype) - :arg default_mimetype: when no mimetype is specified by the server - response assume this mimetype, defaults to `'application/json'` - :arg max_retries: maximum number of retries before an exception is propagated - :arg send_get_body_as: for GET requests with body this option allows - you to specify an alternate way of execution for environments that - don't support passing bodies with GET requests. If you set this to - 'POST' a POST method will be used instead, if to 'source' then the body - will be serialized and passed as a query parameter `source`. - - Any extra keyword arguments will be passed to the `connection_class` - when creating and instance unless overriden by that connection's - options provided as part of the hosts parameter. - """ - - # serialization config - _serializers = DEFAULT_SERIALIZERS.copy() - # if a serializer has been specified, use it for deserialization as well - _serializers[serializer.mimetype] = serializer - # if custom serializers map has been supplied, override the defaults with it - if serializers: - _serializers.update(serializers) - # create a deserializer with our config - self.deserializer = Deserializer(_serializers, default_mimetype) - - self.max_retries = max_retries - self.send_get_body_as = send_get_body_as - - # data serializer - self.serializer = serializer - - # store all strategies... - self.connection_pool_class = connection_pool_class - self.connection_class = connection_class - - # ...save kwargs to be passed to the connections - self.kwargs = kwargs - self.hosts = hosts - - # ...and instantiate them - self.set_connections(hosts) - # retain the original connection instances for sniffing - self.seed_connections = self.connection_pool.connections[:] - - # sniffing data - self.sniffer_timeout = sniffer_timeout - self.sniff_on_connection_fail = sniff_on_connection_fail - self.last_sniff = time.time() - self.sniff_timeout = sniff_timeout - - # callback to construct host dict from data in /_cluster/nodes - self.host_info_callback = host_info_callback - - if sniff_on_start: - self.sniff_hosts() - - def add_connection(self, host): - """ - Create a new :class:`~elasticsearch.Connection` instance and add it to the pool. - - :arg host: kwargs that will be used to create the instance - """ - self.hosts.append(host) - self.set_connections(self.hosts) - - def set_connections(self, hosts): - """ - Instantiate all the connections and crate new connection pool to hold - them. Tries to identify unchanged hosts and re-use existing - :class:`~elasticsearch.Connection` instances. - - :arg hosts: same as `__init__` - """ - # construct the connections - def _create_connection(host): - # if this is not the initial setup look at the existing connection - # options and identify connections that haven't changed and can be - # kept around. - if hasattr(self, 'connection_pool'): - for (connection, old_host) in self.connection_pool.connection_opts: - if old_host == host: - return connection - - # previously unseen params, create new connection - kwargs = self.kwargs.copy() - kwargs.update(host) - return self.connection_class(**kwargs) - connections = map(_create_connection, hosts) - - # pass the hosts dicts to the connection pool to optionally extract parameters from - self.connection_pool = self.connection_pool_class(list(zip(connections, hosts)), **self.kwargs) - - def get_connection(self): - """ - Retreive a :class:`~elasticsearch.Connection` instance from the - :class:`~elasticsearch.ConnectionPool` instance. - """ - if self.sniffer_timeout: - if time.time() >= self.last_sniff + self.sniffer_timeout: - self.sniff_hosts() - return self.connection_pool.get_connection() - - def sniff_hosts(self): - """ - Obtain a list of nodes from the cluster and create a new connection - pool using the information retrieved. - - To extract the node connection parameters use the `nodes_to_host_callback`. - """ - previous_sniff = self.last_sniff - try: - # reset last_sniff timestamp - self.last_sniff = time.time() - # go through all current connections as well as the - # seed_connections for good measure - for c in self.connection_pool.connections + self.seed_connections: - try: - # use small timeout for the sniffing request, should be a fast api call - _, headers, node_info = c.perform_request('GET', '/_nodes', - timeout=self.sniff_timeout) - node_info = self.deserializer.loads(node_info, headers.get('content-type')) - break - except (ConnectionError, SerializationError): - pass - else: - raise TransportError("N/A", "Enable to sniff hosts.") - except: - # keep the previous value on error - self.last_sniff = previous_sniff - raise - - hosts = [] - address = self.connection_class.transport_schema + '_address' - for n in node_info['nodes'].values(): - match = ADDRESS_RE.search(n.get(address, '')) - if not match: - continue - - host = match.groupdict() - if 'port' in host: - host['port'] = int(host['port']) - host = self.host_info_callback(n, host)
View file
elasticsearch-1.0.0.tar.gz/AUTHORS -> elasticsearch5-5.5.3.tar.gz/AUTHORS
Changed
@@ -1,2 +1,58 @@ Honza Král <honza.kral@gmail.com> - +Jordi Llonch <llonchj@gmail.com> +Rob Hudson <rob@cogit8.org> +Njal Karevoll <njal@karevoll.no> +Boaz Leskes <b.leskes@gmail.com> +Graeme Coupar <grambo@grambo.me.uk> +Murhaf Fares <murhaf.fares@gmail.com> +Hari haran <hariharan.uno@gmail.com> +Richard Boulton <richard@tartarus.org> +Brian Hicks <brian@brianthicks.com> +Karan Gupta <karan.gupta@mindtickle.com> +H. İbrahim Güngör <igungor@gmail.com> +James Yu <jyu@rmn.com> +Andrey Balandin <andrvb@gmail.com> +Marco Hoyer <marco.hoyer@immobilienscout24.de> +Max Kutsevol <max@coolvds.com> +Jan Gaedicke <jan.gaedicke@immobilienscout24.de> +Klaas Bosteels <klaas.bosteels@gmail.com> +starenka <starenka0@gmail.com> +Mathieu Geli <mathieu.geli@gmail.com> +Bruno Renié <brutasse@gmail.com> +Ondrej Sika <ondrej@ondrejsika.com> +Alex Ksikes <alex.ksikes@gmail.com> +Ronan Amicel <ronan.amicel@gmail.com> +speedplane <mes65@cornell.edu> +Corey Farwell <coreyf@rwell.org> +Andrew Snare <andrewsnare@godatadriven.com> +Armagnac <hason.liran@gmail.com> +Alexandru Ghitza <aghitza@alum.mit.edu> +Aarni Koskela <akx@iki.fi> +Michael Schier <schiermike@gmail.com> +Yuri Khrustalev <yuri.khrustalev@gmail.com> +Rémy HUBSCHER <rhubscher@mozilla.com> +j0hnsmith <info@whywouldwe.com> +Julian Mehnle <jmehnle@agari.com> +Steven Moy <smoy@yelp.com> +syllogismos +Magnus Bäck <magnus.back@sonymobile.com> +Marc Abramowitz <marc@marc-abramowitz.com> +J Charitopoulos <icharitopoulos@gmail.com> +Sven Wästlund <sven.wastlund@ericsson.com> +Russell Savage <russell.savage@elastic.co> +Georges Toth <georges.toth@govcert.etat.lu> +Malthe Borch <mborch@gmail.com> +Jim Kelly <pthread1981@gmail.com> +Xiuming Chen <cc@cxm.cc> +David Szotten <davidszotten@gmail.com> +Jorgen Jorgensen <jayjaypy@barskdata.net> +Jason Veatch <jtveatch@gmail.com> +Daniel <dmg.lab@outlook.com> +Emil Hessman <emil@hessman.se> +Dmitry Sadovnychyi <dyriax@gmail.com> +Chris Earle <chris.earle@elastic.co> +Tomas Mozes <tmozes@sygic.com> +Alexei Peters <apeters@fargeo.com> +Michael Gibson <michael.x.gibson@oracle.com> +Darryl Ring <darryl@darrylring.com> +Will McGinnis <will@predikto.com>
View file
elasticsearch-1.0.0.tar.gz/CONTRIBUTING.md -> elasticsearch5-5.5.3.tar.gz/CONTRIBUTING.md
Changed
@@ -11,7 +11,7 @@ The process for contributing to any of the Elasticsearch repositories is similar. 1. Please make sure you have signed the [Contributor License - Agreement](http://www.elasticsearch.org/contributor-agreement/). We are not + Agreement](http://www.elastic.co/contributor-agreement/). We are not asking you to assign copyright to us, but to give us the right to distribute your code without restriction. We ask this of all contributors in order to assure our users of the origin and continuing existence of the code. You only @@ -23,7 +23,7 @@ python setup.py test ```` - See the README file in `test_elasticsearch` dirctory for more information on + See the README file in `test_elasticsearch` directory for more information on running the test suite. 3. Rebase your changes. @@ -37,7 +37,7 @@ place, eg “Closes #123″. Please consider adding or modifying tests related to your changes. -Then sit back and wait. There will probably be discussion about the pull +Then sit back and wait. There will probably be a discussion about the pull request and, if any changes are needed, we would love to work with you to get your pull request merged into elasticsearch-py.
View file
elasticsearch-1.0.0.tar.gz/Changelog.rst -> elasticsearch5-5.5.3.tar.gz/Changelog.rst
Changed
@@ -3,6 +3,194 @@ Changelog ========= +5.5.3 (2018-06-20) +------------------ + + * update dependencies for urllib3 + +5.5.2 (2018-01-12) +------------------ + + * Allow connection to SSL ES instances without verifying certs using + client key and client cert. + +5.5.1 (2017-11-10) +------------------ + + * added ``scroll_kwargs`` to the ``scan`` helper + +5.5.0 (2017-11-10) +------------------ + + * ``streaming_bulk`` helper now supports retries with incremental backoff + * ``scan`` helper properly checks for successful shards instead of just + checking ``failed`` + * compatible release with elasticsearch ``5.6.4`` + * fix handling of UTF-8 surrogates + +5.4.0 (2017-05-18) +------------------ + + * ``bulk`` helpers now extract ``pipeline`` parameter from the action + dictionary. + +5.3.0 (2017-03-30) +------------------ + +Compatibility with elasticsearch 5.3 + +5.2.0 (2017-02-12) +------------------ + +The client now automatically sends ``Content-Type`` http header set to +``application/json``. If you are explicitly passing in other encoding than +``json`` you need to set the header manually. + +5.1.0 (2017-01-11) +------------------ + + * Fixed sniffing + +5.0.1 (2016-11-02) +------------------ + +Fixed performance regression in ``scan`` helper + +5.0.0 (2016-10-19) +------------------ + +Version compatible with elasticsearch 5.0 + + * when using SSL certificate validation is now on by default. Install + ``certifi`` or supply root certificate bundle. + * ``elasticsearch.trace`` logger now also logs failed requests, signature of + internal logging method ``log_request_fail`` has changed, all custom + connection classes need to be updated + * added ``headers`` arg to connections to support custom http headers + * passing in a keyword parameter with ``None`` as value will cause that param + to be ignored + +2.4.0 (2016-08-17) +------------------ + + * ``ping`` now ignores all ``TransportError`` exceptions and just returns + ``False`` + * expose ``scroll_id`` on ``ScanError`` + * increase default size for ``scan`` helper to 1000 + +Internal: + + * changed ``Transport.perform_request`` to just return the body, not status as well. + +2.3.0 (2016-02-29) +------------------ + + * added ``client_key`` argument to configure client certificates + * debug logging now includes response body even for failed requests + +2.2.0 (2016-01-05) +------------------ + +Due to change in json encoding the client will no longer mask issues with +encoding - if you work with non-ascii data in python 2 you must use the +``unicode`` type or have proper encoding set in your environment. + + * adding additional options for ssh - ``ssl_assert_hostname`` and + ``ssl_assert_fingerprint`` to the default connection class + * fix sniffing + +2.1.0 (2015-10-19) +------------------ + + * move multiprocessing import inside parallel bulk for Google App Engine + +2.0.0 (2015-10-14) +------------------ + + * Elasticsearch 2.0 compatibility release + +1.8.0 (2015-10-14) +------------------ + + * removed thrift and memcached connections, if you wish to continue using + those, extract the classes and use them separately. + * added a new, parallel version of the bulk helper using thread pools + * In helpers, removed ``bulk_index`` as an alias for ``bulk``. Use ``bulk`` + instead. + +1.7.0 (2015-09-21) +------------------ + + * elasticsearch 2.0 compatibility + * thrift now deprecated, to be removed in future version + * make sure urllib3 always uses keep-alive + +1.6.0 (2015-06-10) +------------------ + + * Add ``indices.flush_synced`` API + * ``helpers.reindex`` now supports reindexing parent/child documents + +1.5.0 (2015-05-18) +------------------ + + * Add support for ``query_cache`` parameter when searching + * helpers have been made more secure by changing defaults to raise an + exception on errors + * removed deprecated options ``replication`` and the deprecated benchmark api. + * Added ``AddonClient`` class to allow for extending the client from outside + +1.4.0 (2015-02-11) +------------------ + + * Using insecure SSL configuration (``verify_cert=False``) raises a warning + * ``reindex`` accepts a ``query`` parameter + * enable ``reindex`` helper to accept any kwargs for underlying ``bulk`` and + ``scan`` calls + * when doing an initial sniff (via ``sniff_on_start``) ignore special sniff timeout + * option to treat ``TransportError`` as normal failure in ``bulk`` helpers + * fixed an issue with sniffing when only a single host was passed in + +1.3.0 (2014-12-31) +------------------ + + * Timeout now doesn't trigger a retry by default (can be overriden by setting + ``retry_on_timeout=True``) + * Introduced new parameter ``retry_on_status`` (defaulting to ``(503, 504, + )``) controls which http status code should lead to a retry. + * Implemented url parsing according to RFC-1738 + * Added support for proper SSL certificate handling + * Required parameters are now checked for non-empty values + * ConnectionPool now checks if any connections were defined + * DummyConnectionPool introduced when no load balancing is needed (only one + connection defined) + * Fixed a race condition in ConnectionPool + +1.2.0 (2014-08-03) +------------------ + +Compatibility with newest (1.3) Elasticsearch APIs. + + * Filter out master-only nodes when sniffing + * Improved docs and error messages + +1.1.1 (2014-07-04) +------------------ + +Bugfix release fixing escaping issues with ``request_timeout``. + +1.1.0 (2014-07-02) +------------------ + +Compatibility with newest Elasticsearch APIs. + + * Test helpers - ``ElasticsearchTestCase`` and ``get_test_client`` for use in your + tests + * Python 3.2 compatibility + * Use ``simplejson`` if installed instead of stdlib json library + * Introducing a global ``request_timeout`` parameter for per-call timeout + * Bug fixes + 1.0.0 (2014-02-11) ------------------ @@ -10,42 +198,42 @@ compatible with 0.90 elasticsearch. * major breaking change - compatible with 1.0 elasticsearch releases only! - * Add an option to change the timeout used for sniff requests (`sniff_timeout`).
View file
elasticsearch-1.0.0.tar.gz/PKG-INFO -> elasticsearch5-5.5.3.tar.gz/PKG-INFO
Changed
@@ -1,36 +1,48 @@ -Metadata-Version: 1.0 -Name: elasticsearch -Version: 1.0.0 +Metadata-Version: 2.1 +Name: elasticsearch5 +Version: 5.5.3 Summary: Python client for Elasticsearch -Home-page: https://github.com/elasticsearch/elasticsearch-py -Author: Honza Král -Author-email: honza.kral@gmail.com +Home-page: https://github.com/elastic/elasticsearch-py +Author: Honza Král, Nick Lang +Author-email: honza.kral@gmail.com, nick@nicklang.com License: Apache License, Version 2.0 Description: Python Elasticsearch Client =========================== - Official low-level client for Elasticsearch. It's goal is to provide common + Official low-level client for Elasticsearch. Its goal is to provide common ground for all Elasticsearch-related code in Python; because of this it tries to be opinion-free and very extendable. - Versioning - ---------- + For a more high level client library with more limited scope, have a look at + `elasticsearch-dsl`_ - a more pythonic library sitting on top of + ``elasticsearch-py``. - There are two branches for development - `master` and `0.4`. Master branch is - used to track all the changes for Elasticsearch 1.0 and beyond whereas 0.4 - tracks Elasticsearch 0.90. + It provides a more convenient and idiomatic way to write and manipulate + `queries`_. It stays close to the Elasticsearch JSON DSL, mirroring its + terminology and structure while exposing the whole range of the DSL from Python + either directly using defined classes or a queryset-like expressions. - Releases with major version 1 (1.X.Y) are to be used with Elasticsearch 1.0 and - later, 0.4 releases are meant to work with Elasticsearch 0.90.X. + It also provides an optional `persistence layer`_ for working with documents as + Python objects in an ORM-like fashion: defining mappings, retrieving and saving + documents, wrapping the document data in user-defined classes. + .. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ + .. _queries: https://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html + .. _persistence layer: https://elasticsearch-dsl.readthedocs.io/en/latest/persistence.html#doctype + + Compatibility + ------------- + + The library is compatible with Elasticsearch ``5.x``. For Elasticsearch ``2.x`` + please use ``elasticsearch2`` and for future releases use ``elasticsearcch``. Installation ------------ - Install the `elasticsearch` package with `pip - <https://pypi.python.org/pypi/elasticsearch>`_:: + Install the ``elasticsearch5`` package with `pip + <https://pypi.python.org/pypi/elasticsearch5>`_:: - pip install elasticsearch + pip install elasticsearch5 Example use @@ -44,6 +56,10 @@ # by default we connect to localhost:9200 >>> es = Elasticsearch() + # create an index in elasticsearch, ignore status code 400 (index already exists) + >>> es.indices.create(index='my-index', ignore=400) + {u'acknowledged': True} + # datetimes will be serialized >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()}) {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True} @@ -54,7 +70,7 @@ `Full documentation`_. - .. _Full documentation: http://elasticsearch-py.rtfd.org/ + .. _Full documentation: https://elasticsearch-py.readthedocs.io/ Features @@ -66,9 +82,10 @@ decoded for performance reasons) * configurable automatic discovery of cluster nodes * persistent connections - * load balancing (with pluggable selection strategy) across all availible nodes + * load balancing (with pluggable selection strategy) across all available nodes * failed connection penalization (time based - failed connections won't be retried until a timeout is reached) + * support for ssl and http authentication * thread safety * pluggable architecture @@ -76,7 +93,7 @@ License ------- - Copyright 2013 Elasticsearch + Copyright 2017 Elasticsearch Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -93,18 +110,23 @@ Build status ------------ - .. image:: https://secure.travis-ci.org/elasticsearch/elasticsearch-py.png - :target: http://travis-ci.org/#!/elasticsearch/elasticsearch-py - - .. image:: https://coveralls.io/repos/elasticsearch/elasticsearch-py/badge.png?branch=master - :target: https://coveralls.io/r/elasticsearch/elasticsearch-py + .. image:: https://secure.travis-ci.org/elastic/elasticsearch-py.png + :target: http://travis-ci.org/#!/elastic/elasticsearch-py Platform: UNKNOWN -Classifier: Development Status :: 4 - Beta +Classifier: Development Status :: 5 - Production/Stable Classifier: License :: OSI Approved :: Apache Software License Classifier: Intended Audience :: Developers Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.6 +Classifier: Programming Language :: Python :: 2.7 Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.2 +Classifier: Programming Language :: Python :: 3.3 +Classifier: Programming Language :: Python :: 3.4 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy +Provides-Extra: develop
View file
elasticsearch-1.0.0.tar.gz/README -> elasticsearch5-5.5.3.tar.gz/README
Changed
@@ -1,28 +1,40 @@ Python Elasticsearch Client =========================== -Official low-level client for Elasticsearch. It's goal is to provide common +Official low-level client for Elasticsearch. Its goal is to provide common ground for all Elasticsearch-related code in Python; because of this it tries to be opinion-free and very extendable. -Versioning ----------- +For a more high level client library with more limited scope, have a look at +`elasticsearch-dsl`_ - a more pythonic library sitting on top of +``elasticsearch-py``. -There are two branches for development - `master` and `0.4`. Master branch is -used to track all the changes for Elasticsearch 1.0 and beyond whereas 0.4 -tracks Elasticsearch 0.90. +It provides a more convenient and idiomatic way to write and manipulate +`queries`_. It stays close to the Elasticsearch JSON DSL, mirroring its +terminology and structure while exposing the whole range of the DSL from Python +either directly using defined classes or a queryset-like expressions. -Releases with major version 1 (1.X.Y) are to be used with Elasticsearch 1.0 and -later, 0.4 releases are meant to work with Elasticsearch 0.90.X. +It also provides an optional `persistence layer`_ for working with documents as +Python objects in an ORM-like fashion: defining mappings, retrieving and saving +documents, wrapping the document data in user-defined classes. +.. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ +.. _queries: https://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html +.. _persistence layer: https://elasticsearch-dsl.readthedocs.io/en/latest/persistence.html#doctype + +Compatibility +------------- + +The library is compatible with Elasticsearch ``5.x``. For Elasticsearch ``2.x`` +please use ``elasticsearch2`` and for future releases use ``elasticsearcch``. Installation ------------ -Install the `elasticsearch` package with `pip -<https://pypi.python.org/pypi/elasticsearch>`_:: +Install the ``elasticsearch5`` package with `pip +<https://pypi.python.org/pypi/elasticsearch5>`_:: - pip install elasticsearch + pip install elasticsearch5 Example use @@ -36,6 +48,10 @@ # by default we connect to localhost:9200 >>> es = Elasticsearch() + # create an index in elasticsearch, ignore status code 400 (index already exists) + >>> es.indices.create(index='my-index', ignore=400) + {u'acknowledged': True} + # datetimes will be serialized >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()}) {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True} @@ -46,7 +62,7 @@ `Full documentation`_. -.. _Full documentation: http://elasticsearch-py.rtfd.org/ +.. _Full documentation: https://elasticsearch-py.readthedocs.io/ Features @@ -58,9 +74,10 @@ decoded for performance reasons) * configurable automatic discovery of cluster nodes * persistent connections - * load balancing (with pluggable selection strategy) across all availible nodes + * load balancing (with pluggable selection strategy) across all available nodes * failed connection penalization (time based - failed connections won't be retried until a timeout is reached) + * support for ssl and http authentication * thread safety * pluggable architecture @@ -68,7 +85,7 @@ License ------- -Copyright 2013 Elasticsearch +Copyright 2017 Elasticsearch Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -85,8 +102,5 @@ Build status ------------ -.. image:: https://secure.travis-ci.org/elasticsearch/elasticsearch-py.png - :target: http://travis-ci.org/#!/elasticsearch/elasticsearch-py - -.. image:: https://coveralls.io/repos/elasticsearch/elasticsearch-py/badge.png?branch=master - :target: https://coveralls.io/r/elasticsearch/elasticsearch-py +.. image:: https://secure.travis-ci.org/elastic/elasticsearch-py.png + :target: http://travis-ci.org/#!/elastic/elasticsearch-py
View file
elasticsearch5-5.5.3.tar.gz/README.rst
Added
@@ -0,0 +1,106 @@ +Python Elasticsearch Client +=========================== + +Official low-level client for Elasticsearch. Its goal is to provide common +ground for all Elasticsearch-related code in Python; because of this it tries +to be opinion-free and very extendable. + +For a more high level client library with more limited scope, have a look at +`elasticsearch-dsl`_ - a more pythonic library sitting on top of +``elasticsearch-py``. + +It provides a more convenient and idiomatic way to write and manipulate +`queries`_. It stays close to the Elasticsearch JSON DSL, mirroring its +terminology and structure while exposing the whole range of the DSL from Python +either directly using defined classes or a queryset-like expressions. + +It also provides an optional `persistence layer`_ for working with documents as +Python objects in an ORM-like fashion: defining mappings, retrieving and saving +documents, wrapping the document data in user-defined classes. + +.. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ +.. _queries: https://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html +.. _persistence layer: https://elasticsearch-dsl.readthedocs.io/en/latest/persistence.html#doctype + +Compatibility +------------- + +The library is compatible with Elasticsearch ``5.x``. For Elasticsearch ``2.x`` +please use ``elasticsearch2`` and for future releases use ``elasticsearcch``. + +Installation +------------ + +Install the ``elasticsearch5`` package with `pip +<https://pypi.python.org/pypi/elasticsearch5>`_:: + + pip install elasticsearch5 + + +Example use +----------- + +Simple use-case:: + + >>> from datetime import datetime + >>> from elasticsearch import Elasticsearch + + # by default we connect to localhost:9200 + >>> es = Elasticsearch() + + # create an index in elasticsearch, ignore status code 400 (index already exists) + >>> es.indices.create(index='my-index', ignore=400) + {u'acknowledged': True} + + # datetimes will be serialized + >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()}) + {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True} + + # but not deserialized + >>> es.get(index="my-index", doc_type="test-type", id=42)['_source'] + {u'any': u'data', u'timestamp': u'2013-05-12T19:45:31.804229'} + +`Full documentation`_. + +.. _Full documentation: https://elasticsearch-py.readthedocs.io/ + + +Features +-------- + +The client's features include: + + * translating basic Python data types to and from json (datetimes are not + decoded for performance reasons) + * configurable automatic discovery of cluster nodes + * persistent connections + * load balancing (with pluggable selection strategy) across all available nodes + * failed connection penalization (time based - failed connections won't be + retried until a timeout is reached) + * support for ssl and http authentication + * thread safety + * pluggable architecture + + +License +------- + +Copyright 2017 Elasticsearch + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Build status +------------ + +.. image:: https://secure.travis-ci.org/elastic/elasticsearch-py.png + :target: http://travis-ci.org/#!/elastic/elasticsearch-py
View file
elasticsearch5-5.5.3.tar.gz/docs/Changelog.rst
Added
@@ -0,0 +1,241 @@ +.. _changelog: + +Changelog +========= + +5.5.3 (2018-06-20) +------------------ + + * update dependencies for urllib3 + +5.5.2 (2018-01-12) +------------------ + + * Allow connection to SSL ES instances without verifying certs using + client key and client cert. + +5.5.1 (2017-11-10) +------------------ + + * added ``scroll_kwargs`` to the ``scan`` helper + +5.5.0 (2017-11-10) +------------------ + + * ``streaming_bulk`` helper now supports retries with incremental backoff + * ``scan`` helper properly checks for successful shards instead of just + checking ``failed`` + * compatible release with elasticsearch ``5.6.4`` + * fix handling of UTF-8 surrogates + +5.4.0 (2017-05-18) +------------------ + + * ``bulk`` helpers now extract ``pipeline`` parameter from the action + dictionary. + +5.3.0 (2017-03-30) +------------------ + +Compatibility with elasticsearch 5.3 + +5.2.0 (2017-02-12) +------------------ + +The client now automatically sends ``Content-Type`` http header set to +``application/json``. If you are explicitly passing in other encoding than +``json`` you need to set the header manually. + +5.1.0 (2017-01-11) +------------------ + + * Fixed sniffing + +5.0.1 (2016-11-02) +------------------ + +Fixed performance regression in ``scan`` helper + +5.0.0 (2016-10-19) +------------------ + +Version compatible with elasticsearch 5.0 + + * when using SSL certificate validation is now on by default. Install + ``certifi`` or supply root certificate bundle. + * ``elasticsearch.trace`` logger now also logs failed requests, signature of + internal logging method ``log_request_fail`` has changed, all custom + connection classes need to be updated + * added ``headers`` arg to connections to support custom http headers + * passing in a keyword parameter with ``None`` as value will cause that param + to be ignored + +2.4.0 (2016-08-17) +------------------ + + * ``ping`` now ignores all ``TransportError`` exceptions and just returns + ``False`` + * expose ``scroll_id`` on ``ScanError`` + * increase default size for ``scan`` helper to 1000 + +Internal: + + * changed ``Transport.perform_request`` to just return the body, not status as well. + +2.3.0 (2016-02-29) +------------------ + + * added ``client_key`` argument to configure client certificates + * debug logging now includes response body even for failed requests + +2.2.0 (2016-01-05) +------------------ + +Due to change in json encoding the client will no longer mask issues with +encoding - if you work with non-ascii data in python 2 you must use the +``unicode`` type or have proper encoding set in your environment. + + * adding additional options for ssh - ``ssl_assert_hostname`` and + ``ssl_assert_fingerprint`` to the default connection class + * fix sniffing + +2.1.0 (2015-10-19) +------------------ + + * move multiprocessing import inside parallel bulk for Google App Engine + +2.0.0 (2015-10-14) +------------------ + + * Elasticsearch 2.0 compatibility release + +1.8.0 (2015-10-14) +------------------ + + * removed thrift and memcached connections, if you wish to continue using + those, extract the classes and use them separately. + * added a new, parallel version of the bulk helper using thread pools + * In helpers, removed ``bulk_index`` as an alias for ``bulk``. Use ``bulk`` + instead. + +1.7.0 (2015-09-21) +------------------ + + * elasticsearch 2.0 compatibility + * thrift now deprecated, to be removed in future version + * make sure urllib3 always uses keep-alive + +1.6.0 (2015-06-10) +------------------ + + * Add ``indices.flush_synced`` API + * ``helpers.reindex`` now supports reindexing parent/child documents + +1.5.0 (2015-05-18) +------------------ + + * Add support for ``query_cache`` parameter when searching + * helpers have been made more secure by changing defaults to raise an + exception on errors + * removed deprecated options ``replication`` and the deprecated benchmark api. + * Added ``AddonClient`` class to allow for extending the client from outside + +1.4.0 (2015-02-11) +------------------ + + * Using insecure SSL configuration (``verify_cert=False``) raises a warning + * ``reindex`` accepts a ``query`` parameter + * enable ``reindex`` helper to accept any kwargs for underlying ``bulk`` and + ``scan`` calls + * when doing an initial sniff (via ``sniff_on_start``) ignore special sniff timeout + * option to treat ``TransportError`` as normal failure in ``bulk`` helpers + * fixed an issue with sniffing when only a single host was passed in + +1.3.0 (2014-12-31) +------------------ + + * Timeout now doesn't trigger a retry by default (can be overriden by setting + ``retry_on_timeout=True``) + * Introduced new parameter ``retry_on_status`` (defaulting to ``(503, 504, + )``) controls which http status code should lead to a retry. + * Implemented url parsing according to RFC-1738 + * Added support for proper SSL certificate handling + * Required parameters are now checked for non-empty values + * ConnectionPool now checks if any connections were defined + * DummyConnectionPool introduced when no load balancing is needed (only one + connection defined) + * Fixed a race condition in ConnectionPool + +1.2.0 (2014-08-03) +------------------ + +Compatibility with newest (1.3) Elasticsearch APIs. + + * Filter out master-only nodes when sniffing + * Improved docs and error messages + +1.1.1 (2014-07-04) +------------------ + +Bugfix release fixing escaping issues with ``request_timeout``. + +1.1.0 (2014-07-02) +------------------ + +Compatibility with newest Elasticsearch APIs. + + * Test helpers - ``ElasticsearchTestCase`` and ``get_test_client`` for use in your + tests + * Python 3.2 compatibility + * Use ``simplejson`` if installed instead of stdlib json library + * Introducing a global ``request_timeout`` parameter for per-call timeout + * Bug fixes + +1.0.0 (2014-02-11) +------------------ + +Elasticsearch 1.0 compatibility. See 0.4.X releases (and 0.4 branch) for code +compatible with 0.90 elasticsearch. +
View file
elasticsearch-1.0.0.tar.gz/docs/api.rst -> elasticsearch5-5.5.3.tar.gz/docs/api.rst
Changed
@@ -3,21 +3,79 @@ API Documentation ================= -.. note:: - - All the API calls map the raw REST api as closely as possible, including - the distinction between required and optional arguments to the calls. This - means that the code makes distinction between positional and keyword arguments; - we, however, recommend that people use keyword arguments for all calls for - consistency and safety. +All the API calls map the raw REST api as closely as possible, including the +distinction between required and optional arguments to the calls. This means +that the code makes distinction between positional and keyword arguments; we, +however, recommend that people **use keyword arguments for all calls for +consistency and safety**. .. note:: - + for compatibility with the Python ecosystem we use ``from_`` instead of ``from`` and ``doc_type`` instead of ``type`` as parameter names. + +Global options +-------------- + +Some parameters are added by the client itself and can be used in all API +calls. + +Ignore +~~~~~~ + +An API call is considered successful (and will return a response) if +elasticsearch returns a 2XX response. Otherwise an instance of +:class:`~elasticsearch.TransportError` (or a more specific subclass) will be +raised. You can see other exception and error states in :ref:`exceptions`. If +you do not wish an exception to be raised you can always pass in an ``ignore`` +parameter with either a single status code that should be ignored or a list of +them:: + + from elasticsearch import Elasticsearch + es = Elasticsearch() + + # ignore 400 cause by IndexAlreadyExistsException when creating an index + es.indices.create(index='test-index', ignore=400) + + # ignore 404 and 400 + es.indices.delete(index='test-index', ignore=[400, 404]) + + +Timeout +~~~~~~~ + +Global timeout can be set when constructing the client (see +:class:`~elasticsearch.Connection`'s ``timeout`` parameter) or on a per-request +basis using ``request_timeout`` (float value in seconds) as part of any API +call, this value will get passed to the ``perform_request`` method of the +connection class:: + + # only wait for 1 second, regardless of the client's default + es.cluster.health(wait_for_status='yellow', request_timeout=1) + +.. note:: + + Some API calls also accept a ``timeout`` parameter that is passed to + Elasticsearch server. This timeout is internal and doesn't guarantee that the + request will end in the specified time. + + .. py:module:: elasticsearch +Response Filtering +~~~~~~~~~~~~~~~~~~ + +The ``filter_path`` parameter is used to reduce the response returned by +elasticsearch. For example, to only return ``_id`` and ``_type``, do:: + + es.search(index='test-index', filter_path=['hits.hits._id', 'hits.hits._type']) + +It also supports the ``*`` wildcard character to match any field or part of a +field's name:: + + es.search(index='test-index', filter_path=['hits.hits._*']) + Elasticsearch ------------- @@ -32,6 +90,12 @@ .. autoclass:: IndicesClient :members: +Ingest +------ + +.. autoclass:: IngestClient + :members: + Cluster ------- @@ -51,8 +115,13 @@ :members: Snapshot ---- +-------- .. autoclass:: SnapshotClient :members: +Tasks +----- + +.. autoclass:: TasksClient + :members:
View file
elasticsearch-1.0.0.tar.gz/docs/conf.py -> elasticsearch5-5.5.3.tar.gz/docs/conf.py
Changed
@@ -49,10 +49,12 @@ # |version| and |release|, also used in various other places throughout the # built documents. # + +import elasticsearch5 # The short X.Y version. -version = '1.0.0' +version = elasticsearch5.__versionstr__ # The full version, including alpha/beta/rc tags. -release = '1.0.0' +release = version # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -96,7 +98,17 @@ # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. -html_theme = 'default' + +on_rtd = os.environ.get('READTHEDOCS', None) == 'True' + +if not on_rtd: # only import and set the theme if we're building docs locally + import sphinx_rtd_theme + html_theme = 'sphinx_rtd_theme' + html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the @@ -218,7 +230,7 @@ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - ('index', 'elasticsearch', u'Elasticsearch Documentation', + ('index', 'elasticsearch-py', u'Elasticsearch Documentation', [u'Honza Král'], 1) ]
View file
elasticsearch-1.0.0.tar.gz/docs/connection.rst -> elasticsearch5-5.5.3.tar.gz/docs/connection.rst
Changed
@@ -3,7 +3,7 @@ Connection Layer API ==================== -All of the classes reponsible for handling the connection to the Elasticsearch +All of the classes responsible for handling the connection to the Elasticsearch cluster. The default subclasses used can be overriden by passing parameters to the :class:`~elasticsearch.Elasticsearch` class. All of the arguments to the client will be passed on to :class:`~elasticsearch.Transport`, @@ -11,7 +11,13 @@ For example if you wanted to use your own implementation of the :class:`~elasticsearch.ConnectionSelector` class you can just pass in the -`selector_class` parameter. +``selector_class`` parameter. + +.. note:: + + :class:`~elasticsearch.ConnectionPool` and related options (like + ``selector_class``) will only be used if more than one connection is defined. + Either directly or via the :ref:`sniffing` mechanism. .. py:module:: elasticsearch @@ -36,8 +42,8 @@ :members: -Connection ----------- +Urllib3HttpConnection (default connection_class) +------------------------------------------------ -.. autoclass:: Connection +.. autoclass:: Urllib3HttpConnection :members:
View file
elasticsearch5-5.5.3.tar.gz/docs/exceptions.rst
Added
@@ -0,0 +1,23 @@ +.. _exceptions: + +Exceptions +========== + +.. py:module:: elasticsearch + +.. autoclass:: ImproperlyConfigured + +.. autoclass:: ElasticsearchException + +.. autoclass:: SerializationError(ElasticsearchException) + +.. autoclass:: TransportError(ElasticsearchException) + :members: + +.. autoclass:: ConnectionError(TransportError) +.. autoclass:: ConnectionTimeout(ConnectionError) +.. autoclass:: SSLError(ConnectionError) + +.. autoclass:: NotFoundError(TransportError) +.. autoclass:: ConflictError(TransportError) +.. autoclass:: RequestError(TransportError)
View file
elasticsearch-1.0.0.tar.gz/docs/helpers.rst -> elasticsearch5-5.5.3.tar.gz/docs/helpers.rst
Changed
@@ -7,12 +7,93 @@ API. +Bulk helpers +------------ + +There are several helpers for the ``bulk`` API since it's requirement for +specific formatting and other considerations can make it cumbersome if used directly. + +All bulk helpers accept an instance of ``Elasticsearch`` class and an iterable +``actions`` (any iterable, can also be a generator, which is ideal in most +cases since it will allow you to index large datasets without the need of +loading them into memory). + +The items in the ``action`` iterable should be the documents we wish to index +in several formats. The most common one is the same as returned by +:meth:`~elasticsearch.Elasticsearch.search`, for example: + +.. code:: python + + { + '_index': 'index-name', + '_type': 'document', + '_id': 42, + '_parent': 5, + 'pipeline': 'my-ingest-pipeline', + '_source': { + "title": "Hello World!", + "body": "..." + } + } + +Alternatively, if `_source` is not present, it will pop all metadata fields +from the doc and use the rest as the document data: + +.. code:: python + + { + "_id": 42, + "_parent": 5, + "title": "Hello World!", + "body": "..." + } + +The :meth:`~elasticsearch.Elasticsearch.bulk` api accepts ``index``, ``create``, +``delete``, and ``update`` actions. Use the ``_op_type`` field to specify an +action (``_op_type`` defaults to ``index``): + +.. code:: python + + { + '_op_type': 'delete', + '_index': 'index-name', + '_type': 'document', + '_id': 42, + } + { + '_op_type': 'update', + '_index': 'index-name', + '_type': 'document', + '_id': 42, + 'doc': {'question': 'The life, universe and everything.'} + } + + +.. note:: + + When reading raw json strings from a file, you can also pass them in + directly (without decoding to dicts first). In that case, however, you lose + the ability to specify anything (index, type, even id) on a per-record + basis, all documents will just be sent to elasticsearch to be indexed + as-is. + + .. py:module:: elasticsearch.helpers .. autofunction:: streaming_bulk +.. autofunction:: parallel_bulk + .. autofunction:: bulk + +Scan +---- + .. autofunction:: scan + +Reindex +------- + .. autofunction:: reindex
View file
elasticsearch-1.0.0.tar.gz/docs/index.rst -> elasticsearch5-5.5.3.tar.gz/docs/index.rst
Changed
@@ -1,34 +1,67 @@ Python Elasticsearch Client =========================== -Official low-level client for Elasticsearch. It's goal is to provide common +Official low-level client for Elasticsearch. Its goal is to provide common ground for all Elasticsearch-related code in Python; because of this it tries to be opinion-free and very extendable. +For a more high level client library with more limited scope, have a look at +`elasticsearch-dsl`_ - it is a more pythonic library sitting on top of +``elasticsearch-py``. -Example Usage +.. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ + +Compatibility ------------- +The library is compatible with all Elasticsearch versions since ``0.90.x`` but you +**have to use a matching major version**: -.. testsetup:: +For **Elasticsearch 5.0** and later, use the major version 5 (``5.x.y``) of the +library. - from datetime import datetime - from elasticsearch import Elasticsearch - es = Elasticsearch() - es.delete_index("test_index", ignore_missing=True) +For **Elasticsearch 2.0** and later, use the major version 2 (``2.x.y``) of the +library. + +For **Elasticsearch 1.0** and later, use the major version 1 (``1.x.y``) of the +library. + +For **Elasticsearch 0.90.x**, use a version from ``0.4.x`` releases of the +library. + +The recommended way to set your requirements in your `setup.py` or +`requirements.txt` is:: + + # Elasticsearch 5.x + elasticsearch>=5.0.0,<6.0.0 + + # Elasticsearch 2.x + elasticsearch>=2.0.0,<3.0.0 + + # Elasticsearch 1.x + elasticsearch>=1.0.0,<2.0.0 + + # Elasticsearch 0.90.x + elasticsearch<1.0.0 + +The development is happening on ``master`` and ``2.x`` branches respectively. + +Example Usage +------------- -.. testcode:: +:: + from datetime import datetime from elasticsearch import Elasticsearch es = Elasticsearch() doc = { 'author': 'kimchy', 'text': 'Elasticsearch: cool. bonsai cool.', - 'timestamp': datetime(2010, 10, 10, 10, 10, 10) + 'timestamp': datetime.now(), } res = es.index(index="test-index", doc_type='tweet', id=1, body=doc) - print(res['ok']) + print(res['created']) res = es.get(index="test-index", doc_type='tweet', id=1) print(res['_source']) @@ -40,29 +73,25 @@ for hit in res['hits']['hits']: print("%(timestamp)s %(author)s: %(text)s" % hit["_source"]) -.. testoutput:: - :hide: - - True - {u'text': u'Elasticsearch: cool. bonsai cool.', u'author': u'kimchy', u'timestamp': u'2010-10-10T10:10:10'} - Got 1 Hits: - 2010-10-10T10:10:10 kimchy: Elasticsearch: cool. bonsai cool. Features -------- -This client was designed as very thin wrapper around Elasticseach's REST API to +This client was designed as very thin wrapper around Elasticsearch's REST API to allow for maximum flexibility. This means that there are no opinions in this client; it also means that some of the APIs are a little cumbersome to use from -Python. We have created some :ref:`helpers` to help with this issue. +Python. We have created some :ref:`helpers` to help with this issue as well as +a more high level library (`elasticsearch-dsl`_) on top of this one to provide +a more convenient way of working with Elasticsearch. + +.. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ Persistent Connections ~~~~~~~~~~~~~~~~~~~~~~ ``elasticsearch-py`` uses persistent connections inside of individual connection pools (one per each configured or sniffed node). Out of the box you can choose -to use ``http``, ``thrift`` or an experimental ``memcached`` protocol to -communicate with the elasticsearch nodes. See :ref:`transports` for more +between two ``http`` protocol implementations. See :ref:`transports` for more information. The transport layer will create an instance of the selected connection class @@ -70,7 +99,7 @@ unresponsive (throwing exceptions while connecting to it) it's put on a timeout by the :class:`~elasticsearch.ConnectionPool` class and only returned to the circulation after the timeout is over (or when no live nodes are left). By -default node are randomized before passed into the pool and round-robin +default nodes are randomized before being passed into the pool and round-robin strategy is used for load balancing. You can customize this behavior by passing parameters to the @@ -81,6 +110,23 @@ the default implementation. +Automatic Retries +~~~~~~~~~~~~~~~~~ + +If a connection to a node fails due to connection issues (raises +:class:`~elasticsearch.ConnectionError`) it is considered in faulty state. It +will be placed on hold for ``dead_timeout`` seconds and the request will be +retried on another node. If a connection fails multiple times in a row the +timeout will get progressively larger to avoid hitting a node that's, by all +indication, down. If no live connection is available, the connection that has +the smallest timeout will be used. + +By default retries are not triggered by a timeout +(:class:`~elasticsearch.ConnectionTimeout`), set ``retry_on_timeout`` to +``True`` to also retry on timeouts. + +.. _sniffing: + Sniffing ~~~~~~~~ @@ -100,8 +146,82 @@ es = Elasticsearch(["seed1", "seed2"], sniff_on_start=True) # you can also sniff periodically and/or after failure: - es = Elasticsearch(["seed1", "seed2"], sniff_on_start=True, sniff_on_connection_fail=True, sniffer_timeout=60) + es = Elasticsearch(["seed1", "seed2"], + sniff_on_start=True, + sniff_on_connection_fail=True, + sniffer_timeout=60) + +Thread safety +~~~~~~~~~~~~~ + +The client is thread safe and can be used in a multi threaded environment. Best +practice is to create a single global instance of the client and use it +throughout your application. If your application is long-running consider +turning on :ref:`sniffing` to make sure the client is up to date on the cluster +location. + +By default we allow ``urllib3`` to open up to 10 connections to each node, if +your application calls for more parallelism, use the ``maxsize`` parameter to +raise the limit:: + + # allow up to 25 connections to each node + es = Elasticsearch(["host1", "host2"], maxsize=25) + +.. note:: + Since we use persistent connections throughout the client it means that the + client doesn't tolerate ``fork`` very well. If your application calls for + multiple processes make sure you create a fresh client after call to + ``fork``. Note that Python's ``multiprocessing`` module uses ``fork`` to + create new processes on POSIX systems. + +SSL and Authentication +~~~~~~~~~~~~~~~~~~~~~~ + +You can configure the client to use ``SSL`` for connecting to your +elasticsearch cluster, including certificate verification and http auth:: + + from elasticsearch import Elasticsearch + + # you can use RFC-1738 to specify the url + es = Elasticsearch(['https://user:secret@localhost:443']) + + # ... or specify common parameters as kwargs + + # use certifi for CA certificates + import certifi + + es = Elasticsearch( + ['localhost', 'otherhost'],
View file
elasticsearch-1.0.0.tar.gz/docs/transports.rst -> elasticsearch5-5.5.3.tar.gz/docs/transports.rst
Changed
@@ -5,16 +5,19 @@ List of transport classes that can be used, simply import your choice and pass it to the constructor of :class:`~elasticsearch.Elasticsearch` as -`connection_class`. Note that Thrift and Memcached protocols are experimental -and require a plugin to be installed in your cluster as well as additional -dependencies (`thrift==0.9` and `pylibmc==1.2`). +`connection_class`. Note that the +:class:`~elasticsearch.connection.RequestsHttpConnection` requires ``requests`` +to be installed. -For example to use the thrift connection just import it and use it. The -connection classes are aware of their respective default ports (9500 for -thrift) so there is no need to specify them unless modified:: +For example to use the ``requests``-based connection just import it and use it:: - from elasticsearch import Elasticsearch, ThriftConnection - es = Elasticsearch(connection_class=ThriftConnection) + from elasticsearch import Elasticsearch, RequestsHttpConnection + es = Elasticsearch(connection_class=RequestsHttpConnection) + +The default connection class is based on ``urllib3`` which is more performant +and lightweight than the optional ``requests``-based class. Only use +``RequestsHttpConnection`` if you have need of any of ``requests`` advanced +features like custom auth plugins etc. .. py:module:: elasticsearch.connection @@ -35,15 +38,3 @@ .. autoclass:: RequestsHttpConnection - -ThriftConnection ----------------- - -.. autoclass:: ThriftConnection - - -MemcachedConnection -------------------- - -.. autoclass:: MemcachedConnection -
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5
Added
+(directory)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info
Added
+(directory)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info/PKG-INFO
Added
@@ -0,0 +1,132 @@ +Metadata-Version: 2.1 +Name: elasticsearch5 +Version: 5.5.3 +Summary: Python client for Elasticsearch +Home-page: https://github.com/elastic/elasticsearch-py +Author: Honza Král, Nick Lang +Author-email: honza.kral@gmail.com, nick@nicklang.com +License: Apache License, Version 2.0 +Description: Python Elasticsearch Client + =========================== + + Official low-level client for Elasticsearch. Its goal is to provide common + ground for all Elasticsearch-related code in Python; because of this it tries + to be opinion-free and very extendable. + + For a more high level client library with more limited scope, have a look at + `elasticsearch-dsl`_ - a more pythonic library sitting on top of + ``elasticsearch-py``. + + It provides a more convenient and idiomatic way to write and manipulate + `queries`_. It stays close to the Elasticsearch JSON DSL, mirroring its + terminology and structure while exposing the whole range of the DSL from Python + either directly using defined classes or a queryset-like expressions. + + It also provides an optional `persistence layer`_ for working with documents as + Python objects in an ORM-like fashion: defining mappings, retrieving and saving + documents, wrapping the document data in user-defined classes. + + .. _elasticsearch-dsl: https://elasticsearch-dsl.readthedocs.io/ + .. _queries: https://elasticsearch-dsl.readthedocs.io/en/latest/search_dsl.html + .. _persistence layer: https://elasticsearch-dsl.readthedocs.io/en/latest/persistence.html#doctype + + Compatibility + ------------- + + The library is compatible with Elasticsearch ``5.x``. For Elasticsearch ``2.x`` + please use ``elasticsearch2`` and for future releases use ``elasticsearcch``. + + Installation + ------------ + + Install the ``elasticsearch5`` package with `pip + <https://pypi.python.org/pypi/elasticsearch5>`_:: + + pip install elasticsearch5 + + + Example use + ----------- + + Simple use-case:: + + >>> from datetime import datetime + >>> from elasticsearch import Elasticsearch + + # by default we connect to localhost:9200 + >>> es = Elasticsearch() + + # create an index in elasticsearch, ignore status code 400 (index already exists) + >>> es.indices.create(index='my-index', ignore=400) + {u'acknowledged': True} + + # datetimes will be serialized + >>> es.index(index="my-index", doc_type="test-type", id=42, body={"any": "data", "timestamp": datetime.now()}) + {u'_id': u'42', u'_index': u'my-index', u'_type': u'test-type', u'_version': 1, u'ok': True} + + # but not deserialized + >>> es.get(index="my-index", doc_type="test-type", id=42)['_source'] + {u'any': u'data', u'timestamp': u'2013-05-12T19:45:31.804229'} + + `Full documentation`_. + + .. _Full documentation: https://elasticsearch-py.readthedocs.io/ + + + Features + -------- + + The client's features include: + + * translating basic Python data types to and from json (datetimes are not + decoded for performance reasons) + * configurable automatic discovery of cluster nodes + * persistent connections + * load balancing (with pluggable selection strategy) across all available nodes + * failed connection penalization (time based - failed connections won't be + retried until a timeout is reached) + * support for ssl and http authentication + * thread safety + * pluggable architecture + + + License + ------- + + Copyright 2017 Elasticsearch + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + Build status + ------------ + + .. image:: https://secure.travis-ci.org/elastic/elasticsearch-py.png + :target: http://travis-ci.org/#!/elastic/elasticsearch-py +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: License :: OSI Approved :: Apache Software License +Classifier: Intended Audience :: Developers +Classifier: Operating System :: OS Independent +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.6 +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.2 +Classifier: Programming Language :: Python :: 3.3 +Classifier: Programming Language :: Python :: 3.4 +Classifier: Programming Language :: Python :: 3.5 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: Implementation :: CPython +Classifier: Programming Language :: Python :: Implementation :: PyPy +Provides-Extra: develop
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info/SOURCES.txt
Added
@@ -0,0 +1,47 @@ +AUTHORS +CONTRIBUTING.md +Changelog.rst +LICENSE +MANIFEST.in +README +README.rst +setup.cfg +setup.py +tox.ini +docs/Changelog.rst +docs/Makefile +docs/api.rst +docs/conf.py +docs/connection.rst +docs/exceptions.rst +docs/helpers.rst +docs/index.rst +docs/transports.rst +elasticsearch5/__init__.py +elasticsearch5/compat.py +elasticsearch5/connection_pool.py +elasticsearch5/exceptions.py +elasticsearch5/serializer.py +elasticsearch5/transport.py +elasticsearch5.egg-info/PKG-INFO +elasticsearch5.egg-info/SOURCES.txt +elasticsearch5.egg-info/dependency_links.txt +elasticsearch5.egg-info/requires.txt +elasticsearch5.egg-info/top_level.txt +elasticsearch5/client/__init__.py +elasticsearch5/client/cat.py +elasticsearch5/client/cluster.py +elasticsearch5/client/indices.py +elasticsearch5/client/ingest.py +elasticsearch5/client/nodes.py +elasticsearch5/client/remote.py +elasticsearch5/client/snapshot.py +elasticsearch5/client/tasks.py +elasticsearch5/client/utils.py +elasticsearch5/connection/__init__.py +elasticsearch5/connection/base.py +elasticsearch5/connection/http_requests.py +elasticsearch5/connection/http_urllib3.py +elasticsearch5/connection/pooling.py +elasticsearch5/helpers/__init__.py +elasticsearch5/helpers/test.py \ No newline at end of file
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info/dependency_links.txt
Changed
(renamed from elasticsearch.egg-info/dependency_links.txt)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info/requires.txt
Added
@@ -0,0 +1,11 @@ +urllib3>=1.21.1 + +[develop] +requests<3.0.0,>=2.0.0 +nose +coverage +mock +pyaml +nosexcover +sphinx +sphinx_rtd_theme
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5.egg-info/top_level.txt
Added
@@ -0,0 +1,1 @@ +elasticsearch5
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/__init__.py
Added
@@ -0,0 +1,25 @@ +from __future__ import absolute_import + +VERSION = (5, 5, 3) +__version__ = VERSION +__versionstr__ = '.'.join(map(str, VERSION)) + +import sys + +if (2, 7) <= sys.version_info < (3, 2): + # On Python 2.7 and Python3 < 3.2, install no-op handler to silence + # `No handlers could be found for logger "elasticsearch"` message per + # <https://docs.python.org/2/howto/logging.html#configuring-logging-for-a-library> + import logging + logger = logging.getLogger('elasticsearch') + logger.addHandler(logging.NullHandler()) + +from .client import Elasticsearch +from .transport import Transport +from .connection_pool import ConnectionPool, ConnectionSelector, \ + RoundRobinSelector +from .serializer import JSONSerializer +from .connection import Connection, RequestsHttpConnection, \ + Urllib3HttpConnection +from .exceptions import * +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client
Added
+(directory)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/__init__.py
Added
@@ -0,0 +1,1620 @@ +from __future__ import unicode_literals +import logging + +from ..transport import Transport +from ..exceptions import TransportError +from ..compat import string_types, urlparse, unquote +from .indices import IndicesClient +from .ingest import IngestClient +from .cluster import ClusterClient +from .cat import CatClient +from .nodes import NodesClient +from .remote import RemoteClient +from .snapshot import SnapshotClient +from .tasks import TasksClient +from .utils import query_params, _make_path, SKIP_IN_PATH + +logger = logging.getLogger('elasticsearch') + +def _normalize_hosts(hosts): + """ + Helper function to transform hosts argument to + :class:`~elasticsearch.Elasticsearch` to a list of dicts. + """ + # if hosts are empty, just defer to defaults down the line + if hosts is None: + return [{}] + + # passed in just one string + if isinstance(hosts, string_types): + hosts = [hosts] + + out = [] + # normalize hosts to dicts + for host in hosts: + if isinstance(host, string_types): + if '://' not in host: + host = "//%s" % host + + parsed_url = urlparse(host) + h = {"host": parsed_url.hostname} + + if parsed_url.port: + h["port"] = parsed_url.port + + if parsed_url.scheme == "https": + h['port'] = parsed_url.port or 443 + h['use_ssl'] = True + + if parsed_url.username or parsed_url.password: + h['http_auth'] = '%s:%s' % (unquote(parsed_url.username), + unquote(parsed_url.password)) + + if parsed_url.path and parsed_url.path != '/': + h['url_prefix'] = parsed_url.path + + out.append(h) + else: + out.append(host) + return out + + +class Elasticsearch(object): + """ + Elasticsearch low-level client. Provides a straightforward mapping from + Python to ES REST endpoints. + + The instance has attributes ``cat``, ``cluster``, ``indices``, ``ingest``, + ``nodes``, ``snapshot`` and ``tasks`` that provide access to instances of + :class:`~elasticsearch.client.CatClient`, + :class:`~elasticsearch.client.ClusterClient`, + :class:`~elasticsearch.client.IndicesClient`, + :class:`~elasticsearch.client.IngestClient`, + :class:`~elasticsearch.client.NodesClient`, + :class:`~elasticsearch.client.SnapshotClient` and + :class:`~elasticsearch.client.TasksClient` respectively. This is the + preferred (and only supported) way to get access to those classes and their + methods. + + You can specify your own connection class which should be used by providing + the ``connection_class`` parameter:: + + # create connection to localhost using the ThriftConnection + es = Elasticsearch(connection_class=ThriftConnection) + + If you want to turn on :ref:`sniffing` you have several options (described + in :class:`~elasticsearch.Transport`):: + + # create connection that will automatically inspect the cluster to get + # the list of active nodes. Start with nodes running on 'esnode1' and + # 'esnode2' + es = Elasticsearch( + ['esnode1', 'esnode2'], + # sniff before doing anything + sniff_on_start=True, + # refresh nodes after a node fails to respond + sniff_on_connection_fail=True, + # and also every 60 seconds + sniffer_timeout=60 + ) + + Different hosts can have different parameters, use a dictionary per node to + specify those:: + + # connect to localhost directly and another node using SSL on port 443 + # and an url_prefix. Note that ``port`` needs to be an int. + es = Elasticsearch([ + {'host': 'localhost'}, + {'host': 'othernode', 'port': 443, 'url_prefix': 'es', 'use_ssl': True}, + ]) + + If using SSL, there are several parameters that control how we deal with + certificates (see :class:`~elasticsearch.Urllib3HttpConnection` for + detailed description of the options):: + + es = Elasticsearch( + ['localhost:443', 'other_host:443'], + # turn on SSL + use_ssl=True, + # make sure we verify SSL certificates (off by default) + verify_certs=True, + # provide a path to CA certs on disk + ca_certs='/path/to/CA_certs' + ) + + SSL client authentication is supported + (see :class:`~elasticsearch.Urllib3HttpConnection` for + detailed description of the options):: + + es = Elasticsearch( + ['localhost:443', 'other_host:443'], + # turn on SSL + use_ssl=True, + # make sure we verify SSL certificates (off by default) + verify_certs=True, + # provide a path to CA certs on disk + ca_certs='/path/to/CA_certs', + # PEM formatted SSL client certificate + client_cert='/path/to/clientcert.pem', + # PEM formatted SSL client key + client_key='/path/to/clientkey.pem' + ) + + Alternatively you can use RFC-1738 formatted URLs, as long as they are not + in conflict with other options:: + + es = Elasticsearch( + [ + 'http://user:secret@localhost:9200/', + 'https://user:secret@other_host:443/production' + ], + verify_certs=True + ) + + """ + def __init__(self, hosts=None, transport_class=Transport, **kwargs): + """ + :arg hosts: list of nodes we should connect to. Node should be a + dictionary ({"host": "localhost", "port": 9200}), the entire dictionary + will be passed to the :class:`~elasticsearch.Connection` class as + kwargs, or a string in the format of ``host[:port]`` which will be + translated to a dictionary automatically. If no value is given the + :class:`~elasticsearch.Urllib3HttpConnection` class defaults will be used. + + :arg transport_class: :class:`~elasticsearch.Transport` subclass to use. + + :arg kwargs: any additional arguments will be passed on to the + :class:`~elasticsearch.Transport` class and, subsequently, to the + :class:`~elasticsearch.Connection` instances. + """ + self.transport = transport_class(_normalize_hosts(hosts), **kwargs) + + # namespaced clients for compatibility with API names + self.indices = IndicesClient(self) + self.ingest = IngestClient(self) + self.cluster = ClusterClient(self) + self.cat = CatClient(self) + self.nodes = NodesClient(self) + self.remote = RemoteClient(self) + self.snapshot = SnapshotClient(self) + self.tasks = TasksClient(self) + + def __repr__(self): + try: + # get a lost of all connections + cons = self.transport.hosts + # truncate to 10 if there are too many + if len(cons) > 5: + cons = cons[:5] + ['...'] + return '<Elasticsearch(%r)>' % cons + except: + # probably operating on custom transport and connection_pool, ignore + return super(Elasticsearch, self).__repr__() + + def _bulk_body(self, body): + # if not passed in a string, serialize items and join by newline + if not isinstance(body, string_types): + body = '\n'.join(map(self.transport.serializer.dumps, body)) + + # bulk body must end with a newline
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/cat.py
Added
@@ -0,0 +1,444 @@ +from .utils import NamespacedClient, query_params, _make_path, SKIP_IN_PATH + +class CatClient(NamespacedClient): + @query_params('format', 'h', 'help', 'local', 'master_timeout', 's', 'v') + def aliases(self, name=None, params=None): + """ + + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-alias.html>`_ + + :arg name: A comma-separated list of alias names to return + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', _make_path('_cat', + 'aliases', name), params=params) + + @query_params('bytes', 'format', 'h', 'help', 'local', 'master_timeout', + 's', 'v') + def allocation(self, node_id=None, params=None): + """ + Allocation provides a snapshot of how shards have located around the + cluster and the state of disk usage. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-allocation.html>`_ + + :arg node_id: A comma-separated list of node IDs or names to limit the + returned information + :arg bytes: The unit in which to display byte values, valid choices are: + 'b', 'k', 'kb', 'm', 'mb', 'g', 'gb', 't', 'tb', 'p', 'pb' + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', _make_path('_cat', + 'allocation', node_id), params=params) + + @query_params('format', 'h', 'help', 'local', 'master_timeout', 's', 'v') + def count(self, index=None, params=None): + """ + Count provides quick access to the document count of the entire cluster, + or individual indices. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-count.html>`_ + + :arg index: A comma-separated list of index names to limit the returned + information + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', _make_path('_cat', 'count', + index), params=params) + + @query_params('format', 'h', 'help', 'local', 'master_timeout', 's', 'ts', + 'v') + def health(self, params=None): + """ + health is a terse, one-line representation of the same information from + :meth:`~elasticsearch.client.cluster.ClusterClient.health` API + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-health.html>`_ + + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg ts: Set to false to disable timestamping, default True + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', '/_cat/health', + params=params) + + @query_params('help', 's') + def help(self, params=None): + """ + A simple help for the cat api. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html>`_ + + :arg help: Return help information, default False + :arg s: Comma-separated list of column names or column aliases to sort + by + """ + return self.transport.perform_request('GET', '/_cat', params=params) + + @query_params('bytes', 'format', 'h', 'health', 'help', 'local', + 'master_timeout', 'pri', 's', 'v') + def indices(self, index=None, params=None): + """ + The indices command provides a cross-section of each index. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-indices.html>`_ + + :arg index: A comma-separated list of index names to limit the returned + information + :arg bytes: The unit in which to display byte values, valid choices are: + 'b', 'k', 'm', 'g' + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg health: A health status ("green", "yellow", or "red" to filter only + indices matching the specified health status, default None, valid + choices are: 'green', 'yellow', 'red' + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg pri: Set to true to return stats only for primary shards, default + False + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', _make_path('_cat', + 'indices', index), params=params) + + @query_params('format', 'h', 'help', 'local', 'master_timeout', 's', 'v') + def master(self, params=None): + """ + Displays the master's node ID, bound IP address, and node name. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-master.html>`_ + + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', '/_cat/master', + params=params) + + @query_params('format', 'full_id', 'h', 'help', 'local', 'master_timeout', + 's', 'v') + def nodes(self, params=None): + """ + The nodes command shows the cluster topology. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html>`_ + + :arg format: a short version of the Accept header, e.g. json, yaml + :arg full_id: Return the full node ID instead of the shortened version + (default: false) + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False + """ + return self.transport.perform_request('GET', '/_cat/nodes', + params=params) + + @query_params('bytes', 'format', 'h', 'help', 'master_timeout', 's', 'v') + def recovery(self, index=None, params=None): + """ + recovery is a view of shard replication. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-recovery.html>`_ + + :arg index: A comma-separated list of index names to limit the returned + information + :arg bytes: The unit in which to display byte values, valid choices are: + 'b', 'k', 'kb', 'm', 'mb', 'g', 'gb', 't', 'tb', 'p', 'pb' + :arg format: a short version of the Accept header, e.g. json, yaml + :arg h: Comma-separated list of column names to display + :arg help: Return help information, default False + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg s: Comma-separated list of column names or column aliases to sort + by + :arg v: Verbose mode. Display column headers, default False
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/cluster.py
Added
@@ -0,0 +1,172 @@ +from .utils import NamespacedClient, query_params, _make_path + +class ClusterClient(NamespacedClient): + @query_params('level', 'local', 'master_timeout', 'timeout', + 'wait_for_active_shards', 'wait_for_events', + 'wait_for_no_relocating_shards', 'wait_for_nodes', 'wait_for_status') + def health(self, index=None, params=None): + """ + Get a very simple status on the health of the cluster. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html>`_ + + :arg index: Limit the information returned to a specific index + :arg level: Specify the level of detail for returned information, + default 'cluster', valid choices are: 'cluster', 'indices', 'shards' + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + :arg wait_for_active_shards: Wait until the specified number of shards + is active + :arg wait_for_events: Wait until all currently queued events with the + given priorty are processed, valid choices are: 'immediate', + 'urgent', 'high', 'normal', 'low', 'languid' + :arg wait_for_no_relocating_shards: Whether to wait until there are no + relocating shards in the cluster + :arg wait_for_nodes: Wait until the specified number of nodes is + available + :arg wait_for_status: Wait until cluster is in a specific state, default + None, valid choices are: 'green', 'yellow', 'red' + """ + return self.transport.perform_request('GET', _make_path('_cluster', + 'health', index), params=params) + + @query_params('local', 'master_timeout') + def pending_tasks(self, params=None): + """ + The pending cluster tasks API returns a list of any cluster-level + changes (e.g. create index, update mapping, allocate or fail shard) + which have not yet been executed. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-pending.html>`_ + + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Specify timeout for connection to master + """ + return self.transport.perform_request('GET', + '/_cluster/pending_tasks', params=params) + + @query_params('allow_no_indices', 'expand_wildcards', 'flat_settings', + 'ignore_unavailable', 'local', 'master_timeout') + def state(self, metric=None, index=None, params=None): + """ + Get a comprehensive state information of the whole cluster. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-state.html>`_ + + :arg metric: Limit the information returned to the specified metrics + :arg index: A comma-separated list of index names; use `_all` or empty + string to perform the operation on all indices + :arg allow_no_indices: Whether to ignore if a wildcard indices + expression resolves into no concrete indices. (This includes `_all` + string or when no indices have been specified) + :arg expand_wildcards: Whether to expand wildcard expression to concrete + indices that are open, closed or both., default 'open', valid + choices are: 'open', 'closed', 'none', 'all' + :arg flat_settings: Return settings in flat format (default: false) + :arg ignore_unavailable: Whether specified concrete indices should be + ignored when unavailable (missing or closed) + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Specify timeout for connection to master + """ + if index and not metric: + metric = '_all' + return self.transport.perform_request('GET', _make_path('_cluster', + 'state', metric, index), params=params) + + @query_params('flat_settings', 'human', 'timeout') + def stats(self, node_id=None, params=None): + """ + The Cluster Stats API allows to retrieve statistics from a cluster wide + perspective. The API returns basic index metrics and information about + the current nodes that form the cluster. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-stats.html>`_ + + :arg node_id: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg flat_settings: Return settings in flat format (default: false) + :arg human: Whether to return time and byte values in human-readable + format., default False + :arg timeout: Explicit operation timeout + """ + url = '/_cluster/stats' + if node_id: + url = _make_path('_cluster/stats/nodes', node_id) + return self.transport.perform_request('GET', url, params=params) + + @query_params('dry_run', 'explain', 'master_timeout', 'metric', + 'retry_failed', 'timeout') + def reroute(self, body=None, params=None): + """ + Explicitly execute a cluster reroute allocation command including specific commands. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-reroute.html>`_ + + :arg body: The definition of `commands` to perform (`move`, `cancel`, + `allocate`) + :arg dry_run: Simulate the operation only and return the resulting state + :arg explain: Return an explanation of why the commands can or cannot be + executed + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg metric: Limit the information returned to the specified metrics. + Defaults to all but metadata, valid choices are: '_all', 'blocks', + 'metadata', 'nodes', 'routing_table', 'master_node', 'version' + :arg retry_failed: Retries allocation of shards that are blocked due to + too many subsequent allocation failures + :arg timeout: Explicit operation timeout + """ + return self.transport.perform_request('POST', '/_cluster/reroute', + params=params, body=body) + + @query_params('flat_settings', 'include_defaults', 'master_timeout', + 'timeout') + def get_settings(self, params=None): + """ + Get cluster settings. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html>`_ + + :arg flat_settings: Return settings in flat format (default: false) + :arg include_defaults: Whether to return all default clusters setting., + default False + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + return self.transport.perform_request('GET', '/_cluster/settings', + params=params) + + @query_params('flat_settings', 'master_timeout', 'timeout') + def put_settings(self, body=None, params=None): + """ + Update cluster wide specific settings. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-update-settings.html>`_ + + :arg body: The settings to be updated. Can be either `transient` or + `persistent` (survives cluster restart). + :arg flat_settings: Return settings in flat format (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + return self.transport.perform_request('PUT', '/_cluster/settings', + params=params, body=body) + + @query_params('include_disk_info', 'include_yes_decisions') + def allocation_explain(self, body=None, params=None): + """ + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html>`_ + + :arg body: The index, shard, and primary flag to explain. Empty means + 'explain the first unassigned shard' + :arg include_disk_info: Return information about disk usage and shard + sizes (default: false) + :arg include_yes_decisions: Return 'YES' decisions in explanation + (default: false) + """ + return self.transport.perform_request('GET', + '/_cluster/allocation/explain', params=params, body=body) + +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/indices.py
Added
@@ -0,0 +1,894 @@ +from .utils import NamespacedClient, query_params, _make_path, SKIP_IN_PATH + +class IndicesClient(NamespacedClient): + @query_params('analyzer', 'attributes', 'char_filter', 'explain', 'field', + 'filter', 'format', 'prefer_local', 'text', 'tokenizer') + def analyze(self, index=None, body=None, params=None): + """ + Perform the analysis process on a text and return the tokens breakdown of the text. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html>`_ + + :arg index: The name of the index to scope the operation + :arg body: The text on which the analysis should be performed + :arg analyzer: The name of the analyzer to use + :arg attributes: A comma-separated list of token attributes to output, + this parameter works only with `explain=true` + :arg char_filter: A comma-separated list of character filters to use for + the analysis + :arg explain: With `true`, outputs more advanced details. (default: + false) + :arg field: Use the analyzer configured for this field (instead of + passing the analyzer name) + :arg filter: A comma-separated list of filters to use for the analysis + :arg format: Format of the output, default 'detailed', valid choices + are: 'detailed', 'text' + :arg prefer_local: With `true`, specify that a local shard should be + used if available, with `false`, use a random shard (default: true) + :arg text: The text on which the analysis should be performed (when + request body is not used) + :arg tokenizer: The name of the tokenizer to use for the analysis + """ + return self.transport.perform_request('GET', _make_path(index, + '_analyze'), params=params, body=body) + + @query_params('allow_no_indices', 'expand_wildcards', 'force', + 'ignore_unavailable', 'operation_threading') + def refresh(self, index=None, params=None): + """ + Explicitly refresh one or more index, making all operations performed + since the last refresh available for search. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html>`_ + + :arg index: A comma-separated list of index names; use `_all` or empty + string to perform the operation on all indices + :arg allow_no_indices: Whether to ignore if a wildcard indices + expression resolves into no concrete indices. (This includes `_all` + string or when no indices have been specified) + :arg expand_wildcards: Whether to expand wildcard expression to concrete + indices that are open, closed or both., default 'open', valid + choices are: 'open', 'closed', 'none', 'all' + :arg force: Force a refresh even if not required, default False + :arg ignore_unavailable: Whether specified concrete indices should be + ignored when unavailable (missing or closed) + :arg operation_threading: TODO: ? + """ + return self.transport.perform_request('POST', _make_path(index, + '_refresh'), params=params) + + @query_params('allow_no_indices', 'expand_wildcards', 'force', + 'ignore_unavailable', 'wait_if_ongoing') + def flush(self, index=None, params=None): + """ + Explicitly flush one or more indices. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-flush.html>`_ + + :arg index: A comma-separated list of index names; use `_all` or empty + string for all indices + :arg allow_no_indices: Whether to ignore if a wildcard indices + expression resolves into no concrete indices. (This includes `_all` + string or when no indices have been specified) + :arg expand_wildcards: Whether to expand wildcard expression to concrete + indices that are open, closed or both., default 'open', valid + choices are: 'open', 'closed', 'none', 'all' + :arg force: Whether a flush should be forced even if it is not + necessarily needed ie. if no changes will be committed to the index. + This is useful if transaction log IDs should be incremented even if + no uncommitted changes are present. (This setting can be considered + as internal) + :arg ignore_unavailable: Whether specified concrete indices should be + ignored when unavailable (missing or closed) + :arg wait_if_ongoing: If set to true the flush operation will block + until the flush can be executed if another flush operation is + already executing. The default is true. If set to false the flush + will be skipped iff if another flush operation is already running. + """ + return self.transport.perform_request('POST', _make_path(index, + '_flush'), params=params) + + @query_params('master_timeout', 'timeout', 'update_all_types', + 'wait_for_active_shards') + def create(self, index, body=None, params=None): + """ + Create an index in Elasticsearch. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html>`_ + + :arg index: The name of the index + :arg body: The configuration for the index (`settings` and `mappings`) + :arg master_timeout: Specify timeout for connection to master + :arg timeout: Explicit operation timeout + :arg update_all_types: Whether to update the mapping for all fields with + the same name across all types or not + :arg wait_for_active_shards: Set the number of active shards to wait for + before the operation returns. + """ + if index in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'index'.") + return self.transport.perform_request('PUT', _make_path(index), + params=params, body=body) + + @query_params('allow_no_indices', 'expand_wildcards', 'flat_settings', + 'human', 'ignore_unavailable', 'include_defaults', 'local') + def get(self, index, feature=None, params=None): + """ + The get index API allows to retrieve information about one or more indexes. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-index.html>`_ + + :arg index: A comma-separated list of index names + :arg feature: A comma-separated list of features + :arg allow_no_indices: Ignore if a wildcard expression resolves to no + concrete indices (default: false) + :arg expand_wildcards: Whether wildcard expressions should get expanded + to open or closed indices (default: open), default 'open', valid + choices are: 'open', 'closed', 'none', 'all' + :arg flat_settings: Return settings in flat format (default: false) + :arg human: Whether to return version and creation date values in human- + readable format., default False + :arg ignore_unavailable: Ignore unavailable indexes (default: false) + :arg include_defaults: Whether to return all default setting for each of + the indices., default False + :arg local: Return local information, do not retrieve the state from + master node (default: false) + """ + if index in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'index'.") + return self.transport.perform_request('GET', _make_path(index, + feature), params=params) + + @query_params('allow_no_indices', 'expand_wildcards', 'ignore_unavailable', + 'master_timeout', 'timeout') + def open(self, index, params=None): + """ + Open a closed index to make it available for search. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-open-close.html>`_ + + :arg index: The name of the index + :arg allow_no_indices: Whether to ignore if a wildcard indices + expression resolves into no concrete indices. (This includes `_all` + string or when no indices have been specified) + :arg expand_wildcards: Whether to expand wildcard expression to concrete + indices that are open, closed or both., default 'closed', valid + choices are: 'open', 'closed', 'none', 'all' + :arg ignore_unavailable: Whether specified concrete indices should be + ignored when unavailable (missing or closed) + :arg master_timeout: Specify timeout for connection to master + :arg timeout: Explicit operation timeout + """ + if index in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'index'.") + return self.transport.perform_request('POST', _make_path(index, + '_open'), params=params) + + @query_params('allow_no_indices', 'expand_wildcards', 'ignore_unavailable', + 'master_timeout', 'timeout') + def close(self, index, params=None): + """ + Close an index to remove it's overhead from the cluster. Closed index + is blocked for read/write operations. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-open-close.html>`_ + + :arg index: The name of the index + :arg allow_no_indices: Whether to ignore if a wildcard indices + expression resolves into no concrete indices. (This includes `_all` + string or when no indices have been specified) + :arg expand_wildcards: Whether to expand wildcard expression to concrete + indices that are open, closed or both., default 'open', valid + choices are: 'open', 'closed', 'none', 'all' + :arg ignore_unavailable: Whether specified concrete indices should be + ignored when unavailable (missing or closed) + :arg master_timeout: Specify timeout for connection to master + :arg timeout: Explicit operation timeout + """ + if index in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'index'.") + return self.transport.perform_request('POST', _make_path(index, + '_close'), params=params) + + @query_params('master_timeout', 'timeout') + def delete(self, index, params=None): + """ + Delete an index in Elasticsearch + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html>`_ + + :arg index: A comma-separated list of indices to delete; use `_all` or + `*` string to delete all indices + :arg master_timeout: Specify timeout for connection to master + :arg timeout: Explicit operation timeout + """ + if index in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'index'.") + return self.transport.perform_request('DELETE', _make_path(index),
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/ingest.py
Added
@@ -0,0 +1,61 @@ +from .utils import NamespacedClient, query_params, _make_path, SKIP_IN_PATH + +class IngestClient(NamespacedClient): + @query_params('master_timeout') + def get_pipeline(self, id=None, params=None): + """ + `<https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest.html>`_ + + :arg id: Comma separated list of pipeline ids. Wildcards supported + :arg master_timeout: Explicit operation timeout for connection to master + node + """ + return self.transport.perform_request('GET', _make_path('_ingest', + 'pipeline', id), params=params) + + @query_params('master_timeout', 'timeout') + def put_pipeline(self, id, body, params=None): + """ + `<https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest.html>`_ + + :arg id: Pipeline ID + :arg body: The ingest definition + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + for param in (id, body): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('PUT', _make_path('_ingest', + 'pipeline', id), params=params, body=body) + + @query_params('master_timeout', 'timeout') + def delete_pipeline(self, id, params=None): + """ + `<https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest.html>`_ + + :arg id: Pipeline ID + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + if id in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'id'.") + return self.transport.perform_request('DELETE', _make_path('_ingest', + 'pipeline', id), params=params) + + @query_params('verbose') + def simulate(self, body, id=None, params=None): + """ + `<https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest.html>`_ + + :arg body: The simulate definition + :arg id: Pipeline ID + :arg verbose: Verbose mode. Display data output for each processor in + executed pipeline, default False + """ + if body in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'body'.") + return self.transport.perform_request('GET', _make_path('_ingest', + 'pipeline', id, '_simulate'), params=params, body=body)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/nodes.py
Added
@@ -0,0 +1,86 @@ +from .utils import NamespacedClient, query_params, _make_path + +class NodesClient(NamespacedClient): + @query_params('flat_settings', 'timeout') + def info(self, node_id=None, metric=None, params=None): + """ + The cluster nodes info API allows to retrieve one or more (or all) of + the cluster nodes information. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-info.html>`_ + + :arg node_id: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg metric: A comma-separated list of metrics you wish returned. Leave + empty to return all. + :arg flat_settings: Return settings in flat format (default: false) + :arg timeout: Explicit operation timeout + """ + return self.transport.perform_request('GET', _make_path('_nodes', + node_id, metric), params=params) + + @query_params('completion_fields', 'fielddata_fields', 'fields', 'groups', + 'include_segment_file_sizes', 'level', 'timeout', 'types') + def stats(self, node_id=None, metric=None, index_metric=None, params=None): + """ + The cluster nodes stats API allows to retrieve one or more (or all) of + the cluster nodes statistics. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html>`_ + + :arg node_id: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg metric: Limit the information returned to the specified metrics + :arg index_metric: Limit the information returned for `indices` metric + to the specific index metrics. Isn't used if `indices` (or `all`) + metric isn't specified. + :arg completion_fields: A comma-separated list of fields for `fielddata` + and `suggest` index metric (supports wildcards) + :arg fielddata_fields: A comma-separated list of fields for `fielddata` + index metric (supports wildcards) + :arg fields: A comma-separated list of fields for `fielddata` and + `completion` index metric (supports wildcards) + :arg groups: A comma-separated list of search groups for `search` index + metric + :arg include_segment_file_sizes: Whether to report the aggregated disk + usage of each one of the Lucene index files (only applies if segment + stats are requested), default False + :arg level: Return indices stats aggregated at index, node or shard + level, default 'node', valid choices are: 'indices', 'node', + 'shards' + :arg timeout: Explicit operation timeout + :arg types: A comma-separated list of document types for the `indexing` + index metric + """ + return self.transport.perform_request('GET', _make_path('_nodes', + node_id, 'stats', metric, index_metric), params=params) + + @query_params('type', 'ignore_idle_threads', 'interval', 'snapshots', + 'threads', 'timeout') + def hot_threads(self, node_id=None, params=None): + """ + An API allowing to get the current hot threads on each node in the cluster. + `<https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html>`_ + + :arg node_id: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg type: The type to sample (default: cpu), valid choices are: + 'cpu', 'wait', 'block' + :arg ignore_idle_threads: Don't show threads that are in known-idle + places, such as waiting on a socket select or pulling from an empty + task queue (default: true) + :arg interval: The interval for the second sampling of threads + :arg snapshots: Number of samples of thread stacktrace (default: 10) + :arg threads: Specify the number of threads to provide information for + (default: 3) + :arg timeout: Explicit operation timeout + """ + # avoid python reserved words + if params and 'type_' in params: + params['type'] = params.pop('type_') + return self.transport.perform_request('GET', _make_path('_cluster', + 'nodes', node_id, 'hotthreads'), params=params)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/remote.py
Added
@@ -0,0 +1,11 @@ +from .utils import NamespacedClient, query_params, _make_path, SKIP_IN_PATH + +class RemoteClient(NamespacedClient): + @query_params() + def info(self, params=None): + """ + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-remote-info.html>`_ + """ + return self.transport.perform_request('GET', '/_remote/info', + params=params) +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/snapshot.py
Added
@@ -0,0 +1,163 @@ +from .utils import NamespacedClient, query_params, _make_path, SKIP_IN_PATH + +class SnapshotClient(NamespacedClient): + @query_params('master_timeout', 'wait_for_completion') + def create(self, repository, snapshot, body=None, params=None): + """ + Create a snapshot in repository + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg snapshot: A snapshot name + :arg body: The snapshot definition + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg wait_for_completion: Should this request wait until the operation + has completed before returning, default False + """ + for param in (repository, snapshot): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('PUT', _make_path('_snapshot', + repository, snapshot), params=params, body=body) + + @query_params('master_timeout') + def delete(self, repository, snapshot, params=None): + """ + Deletes a snapshot from a repository. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg snapshot: A snapshot name + :arg master_timeout: Explicit operation timeout for connection to master + node + """ + for param in (repository, snapshot): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('DELETE', + _make_path('_snapshot', repository, snapshot), params=params) + + @query_params('ignore_unavailable', 'master_timeout') + def get(self, repository, snapshot, params=None): + """ + Retrieve information about a snapshot. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg snapshot: A comma-separated list of snapshot names + :arg ignore_unavailable: Whether to ignore unavailable snapshots, + defaults to false which means a SnapshotMissingException is thrown + :arg master_timeout: Explicit operation timeout for connection to master + node + """ + for param in (repository, snapshot): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('GET', _make_path('_snapshot', + repository, snapshot), params=params) + + @query_params('master_timeout', 'timeout') + def delete_repository(self, repository, params=None): + """ + Removes a shared file system repository. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A comma-separated list of repository names + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + if repository in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'repository'.") + return self.transport.perform_request('DELETE', + _make_path('_snapshot', repository), params=params) + + @query_params('local', 'master_timeout') + def get_repository(self, repository=None, params=None): + """ + Return information about registered repositories. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A comma-separated list of repository names + :arg local: Return local information, do not retrieve the state from + master node (default: false) + :arg master_timeout: Explicit operation timeout for connection to master + node + """ + return self.transport.perform_request('GET', _make_path('_snapshot', + repository), params=params) + + @query_params('master_timeout', 'timeout', 'verify') + def create_repository(self, repository, body, params=None): + """ + Registers a shared file system repository. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg body: The repository definition + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + :arg verify: Whether to verify the repository after creation + """ + for param in (repository, body): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('PUT', _make_path('_snapshot', + repository), params=params, body=body) + + @query_params('master_timeout', 'wait_for_completion') + def restore(self, repository, snapshot, body=None, params=None): + """ + Restore a snapshot. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg snapshot: A snapshot name + :arg body: Details of what to restore + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg wait_for_completion: Should this request wait until the operation + has completed before returning, default False + """ + for param in (repository, snapshot): + if param in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument.") + return self.transport.perform_request('POST', _make_path('_snapshot', + repository, snapshot, '_restore'), params=params, body=body) + + @query_params('ignore_unavailable', 'master_timeout') + def status(self, repository=None, snapshot=None, params=None): + """ + Return information about all currently running snapshots. By specifying + a repository name, it's possible to limit the results to a particular + repository. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg snapshot: A comma-separated list of snapshot names + :arg ignore_unavailable: Whether to ignore unavailable snapshots, + defaults to false which means a SnapshotMissingException is thrown + :arg master_timeout: Explicit operation timeout for connection to master + node + """ + return self.transport.perform_request('GET', _make_path('_snapshot', + repository, snapshot, '_status'), params=params) + + @query_params('master_timeout', 'timeout') + def verify_repository(self, repository, params=None): + """ + Returns a list of nodes where repository was successfully verified or + an error message if verification process failed. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html>`_ + + :arg repository: A repository name + :arg master_timeout: Explicit operation timeout for connection to master + node + :arg timeout: Explicit operation timeout + """ + if repository in SKIP_IN_PATH: + raise ValueError("Empty value passed for a required argument 'repository'.") + return self.transport.perform_request('POST', _make_path('_snapshot', + repository, '_verify'), params=params)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/tasks.py
Added
@@ -0,0 +1,58 @@ +from .utils import NamespacedClient, query_params, _make_path + +class TasksClient(NamespacedClient): + @query_params('actions', 'detailed', 'group_by', 'nodes', 'parent_node', + 'parent_task_id', 'wait_for_completion') + def list(self, params=None): + """ + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html>`_ + + :arg actions: A comma-separated list of actions that should be returned. + Leave empty to return all. + :arg detailed: Return detailed task information (default: false) + :arg group_by: Group tasks by nodes or parent/child relationships, + default 'nodes', valid choices are: 'nodes', 'parents' + :arg nodes: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg parent_node: Return tasks with specified parent node. + :arg parent_task_id: Return tasks with specified parent task id + (node_id:task_number). Set to -1 to return all. + :arg wait_for_completion: Wait for the matching tasks to complete + (default: false) + """ + return self.transport.perform_request('GET', '/_tasks', params=params) + + @query_params('actions', 'nodes', 'parent_task_id') + def cancel(self, task_id=None, params=None): + """ + + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html>`_ + + :arg task_id: Cancel the task with specified task id + (node_id:task_number) + :arg actions: A comma-separated list of actions that should be + cancelled. Leave empty to cancel all. + :arg nodes: A comma-separated list of node IDs or names to limit the + returned information; use `_local` to return information from the + node you're connecting to, leave empty to get information from all + nodes + :arg parent_task_id: Cancel tasks with specified parent task id + (node_id:task_number). Set to -1 to cancel all. + """ + return self.transport.perform_request('POST', _make_path('_tasks', + task_id, '_cancel'), params=params) + + @query_params('wait_for_completion') + def get(self, task_id=None, params=None): + """ + Retrieve information for a particular task. + `<http://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html>`_ + + :arg task_id: Return the task with specified id (node_id:task_number) + :arg wait_for_completion: Wait for the matching tasks to complete + (default: false) + """ + return self.transport.perform_request('GET', _make_path('_tasks', + task_id), params=params)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/client/utils.py
Added
@@ -0,0 +1,91 @@ +from __future__ import unicode_literals + +import weakref +from datetime import date, datetime +from functools import wraps +from ..compat import string_types, quote_plus + +# parts of URL to be omitted +SKIP_IN_PATH = (None, '', b'', [], ()) + +def _escape(value): + """ + Escape a single value of a URL string or a query parameter. If it is a list + or tuple, turn it into a comma-separated string first. + """ + + # make sequences into comma-separated stings + if isinstance(value, (list, tuple)): + value = ','.join(value) + + # dates and datetimes into isoformat + elif isinstance(value, (date, datetime)): + value = value.isoformat() + + # make bools into true/false strings + elif isinstance(value, bool): + value = str(value).lower() + + # don't decode bytestrings + elif isinstance(value, bytes): + return value + + # encode strings to utf-8 + if isinstance(value, string_types): + return value.encode('utf-8') + + return str(value) + +def _make_path(*parts): + """ + Create a URL string from parts, omit all `None` values and empty strings. + Convert lists nad tuples to comma separated values. + """ + #TODO: maybe only allow some parts to be lists/tuples ? + return '/' + '/'.join( + # preserve ',' and '*' in url for nicer URLs in logs + quote_plus(_escape(p), b',*') for p in parts if p not in SKIP_IN_PATH) + +# parameters that apply to all methods +GLOBAL_PARAMS = ('pretty', 'human', 'error_trace', 'format', 'filter_path') + +def query_params(*es_query_params): + """ + Decorator that pops all accepted parameters from method's kwargs and puts + them in the params argument. + """ + def _wrapper(func): + @wraps(func) + def _wrapped(*args, **kwargs): + params = {} + if 'params' in kwargs: + params = kwargs.pop('params').copy() + for p in es_query_params + GLOBAL_PARAMS: + if p in kwargs: + v = kwargs.pop(p) + if v is not None: + params[p] = _escape(v) + + # don't treat ignore and request_timeout as other params to avoid escaping + for p in ('ignore', 'request_timeout'): + if p in kwargs: + params[p] = kwargs.pop(p) + return func(*args, params=params, **kwargs) + return _wrapped + return _wrapper + + +class NamespacedClient(object): + def __init__(self, client): + self.client = client + + @property + def transport(self): + return self.client.transport + +class AddonClient(NamespacedClient): + @classmethod + def infect_client(cls, client): + addon = cls(weakref.proxy(client)) + setattr(client, cls.namespace, addon) + return client
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/compat.py
Added
@@ -0,0 +1,15 @@ +import sys + +PY2 = sys.version_info[0] == 2 + +if PY2: + string_types = basestring, + from urllib import quote_plus, urlencode, unquote + from urlparse import urlparse + from itertools import imap as map + from Queue import Queue +else: + string_types = str, bytes + from urllib.parse import quote_plus, urlencode, urlparse, unquote + map = map + from queue import Queue
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection
Added
+(directory)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection/__init__.py
Added
@@ -0,0 +1,3 @@ +from .base import Connection +from .http_requests import RequestsHttpConnection +from .http_urllib3 import Urllib3HttpConnection
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection/base.py
Added
@@ -0,0 +1,127 @@ +import logging +try: + import simplejson as json +except ImportError: + import json + +from ..exceptions import TransportError, HTTP_EXCEPTIONS + +logger = logging.getLogger('elasticsearch') + +# create the elasticsearch.trace logger, but only set propagate to False if the +# logger hasn't already been configured +_tracer_already_configured = 'elasticsearch.trace' in logging.Logger.manager.loggerDict +tracer = logging.getLogger('elasticsearch.trace') +if not _tracer_already_configured: + tracer.propagate = False + + +class Connection(object): + """ + Class responsible for maintaining a connection to an Elasticsearch node. It + holds persistent connection pool to it and it's main interface + (`perform_request`) is thread-safe. + + Also responsible for logging. + """ + def __init__(self, host='localhost', port=9200, use_ssl=False, url_prefix='', timeout=10, **kwargs): + """ + :arg host: hostname of the node (default: localhost) + :arg port: port to use (integer, default: 9200) + :arg url_prefix: optional url prefix for elasticsearch + :arg timeout: default timeout in seconds (float, default: 10) + """ + scheme = kwargs.get('scheme', 'http') + if use_ssl or scheme == 'https': + scheme = 'https' + use_ssl = True + self.use_ssl = use_ssl + + self.host = '%s://%s:%s' % (scheme, host, port) + if url_prefix: + url_prefix = '/' + url_prefix.strip('/') + self.url_prefix = url_prefix + self.timeout = timeout + + def __repr__(self): + return '<%s: %s>' % (self.__class__.__name__, self.host) + + def _pretty_json(self, data): + # pretty JSON in tracer curl logs + try: + return json.dumps(json.loads(data), sort_keys=True, indent=2, separators=(',', ': ')).replace("'", r'\u0027') + except (ValueError, TypeError): + # non-json data or a bulk request + return data + + def _log_trace(self, method, path, body, status_code, response, duration): + if not tracer.isEnabledFor(logging.INFO) or not tracer.handlers: + return + + # include pretty in trace curls + path = path.replace('?', '?pretty&', 1) if '?' in path else path + '?pretty' + if self.url_prefix: + path = path.replace(self.url_prefix, '', 1) + tracer.info("curl %s-X%s 'http://localhost:9200%s' -d '%s'", + "-H 'Content-Type: application/json' " if body else '', + method, path, self._pretty_json(body) if body else '') + + if tracer.isEnabledFor(logging.DEBUG): + tracer.debug('#[%s] (%.3fs)\n#%s', status_code, duration, self._pretty_json(response).replace('\n', '\n#') if response else '') + + def log_request_success(self, method, full_url, path, body, status_code, response, duration): + """ Log a successful API call. """ + # TODO: optionally pass in params instead of full_url and do urlencode only when needed + + # body has already been serialized to utf-8, deserialize it for logging + # TODO: find a better way to avoid (de)encoding the body back and forth + if body: + body = body.decode('utf-8', 'ignore') + + logger.info( + '%s %s [status:%s request:%.3fs]', method, full_url, + status_code, duration + ) + logger.debug('> %s', body) + logger.debug('< %s', response) + + self._log_trace(method, path, body, status_code, response, duration) + + def log_request_fail(self, method, full_url, path, body, duration, status_code=None, response=None, exception=None): + """ Log an unsuccessful API call. """ + # do not log 404s on HEAD requests + if method == 'HEAD' and status_code == 404: + return + logger.warning( + '%s %s [status:%s request:%.3fs]', method, full_url, + status_code or 'N/A', duration, exc_info=exception is not None + ) + + # body has already been serialized to utf-8, deserialize it for logging + # TODO: find a better way to avoid (de)encoding the body back and forth + if body: + body = body.decode('utf-8', 'ignore') + + logger.debug('> %s', body) + + self._log_trace(method, path, body, status_code, response, duration) + + if response is not None: + logger.debug('< %s', response) + + def _raise_error(self, status_code, raw_data): + """ Locate appropriate exception and raise it. """ + error_message = raw_data + additional_info = None + try: + if raw_data: + additional_info = json.loads(raw_data) + error_message = additional_info.get('error', error_message) + if isinstance(error_message, dict) and 'type' in error_message: + error_message = error_message['type'] + except (ValueError, TypeError) as err: + logger.warning('Undecodable raw error response from server: %s', err) + + raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) + +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection/http_requests.py
Added
@@ -0,0 +1,100 @@ +import time +import warnings +try: + import requests + REQUESTS_AVAILABLE = True +except ImportError: + REQUESTS_AVAILABLE = False + +from .base import Connection +from ..exceptions import ConnectionError, ImproperlyConfigured, ConnectionTimeout, SSLError +from ..compat import urlencode, string_types + +class RequestsHttpConnection(Connection): + """ + Connection using the `requests` library. + + :arg http_auth: optional http auth information as either ':' separated + string or a tuple. Any value will be passed into requests as `auth`. + :arg use_ssl: use ssl for the connection if `True` + :arg verify_certs: whether to verify SSL certificates + :arg ca_certs: optional path to CA bundle. By default standard requests' + bundle will be used. + :arg client_cert: path to the file containing the private key and the + certificate, or cert only if using client_key + :arg client_key: path to the file containing the private key if using + separate cert and key files (client_cert will contain only the cert) + :arg headers: any custom http headers to be add to requests + """ + def __init__(self, host='localhost', port=9200, http_auth=None, + use_ssl=False, verify_certs=True, ca_certs=None, client_cert=None, + client_key=None, headers=None, **kwargs): + if not REQUESTS_AVAILABLE: + raise ImproperlyConfigured("Please install requests to use RequestsHttpConnection.") + + super(RequestsHttpConnection, self).__init__(host=host, port=port, use_ssl=use_ssl, **kwargs) + self.session = requests.Session() + self.session.headers = headers or {} + self.session.headers.setdefault('content-type', 'application/json') + if http_auth is not None: + if isinstance(http_auth, (tuple, list)): + http_auth = tuple(http_auth) + elif isinstance(http_auth, string_types): + http_auth = tuple(http_auth.split(':', 1)) + self.session.auth = http_auth + self.base_url = 'http%s://%s:%d%s' % ( + 's' if self.use_ssl else '', + host, port, self.url_prefix + ) + self.session.verify = verify_certs + if not client_key: + self.session.cert = client_cert + elif client_cert: + # cert is a tuple of (certfile, keyfile) + self.session.cert = (client_cert, client_key) + if ca_certs: + if not verify_certs: + raise ImproperlyConfigured("You cannot pass CA certificates when verify SSL is off.") + self.session.verify = ca_certs + + if self.use_ssl and not verify_certs: + warnings.warn( + 'Connecting to %s using SSL with verify_certs=False is insecure.' % self.base_url) + + def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): + url = self.base_url + url + if params: + url = '%s?%s' % (url, urlencode(params or {})) + + start = time.time() + request = requests.Request(method=method, url=url, data=body) + prepared_request = self.session.prepare_request(request) + settings = self.session.merge_environment_settings(prepared_request.url, {}, None, None, None) + send_kwargs = {'timeout': timeout or self.timeout} + send_kwargs.update(settings) + try: + response = self.session.send(prepared_request, **send_kwargs) + duration = time.time() - start + raw_data = response.text + except Exception as e: + self.log_request_fail(method, url, prepared_request.path_url, body, time.time() - start, exception=e) + if isinstance(e, requests.exceptions.SSLError): + raise SSLError('N/A', str(e), e) + if isinstance(e, requests.Timeout): + raise ConnectionTimeout('TIMEOUT', str(e), e) + raise ConnectionError('N/A', str(e), e) + + # raise errors based on http status codes, let the client handle those if needed + if not (200 <= response.status_code < 300) and response.status_code not in ignore: + self.log_request_fail(method, url, response.request.path_url, body, duration, response.status_code, raw_data) + self._raise_error(response.status_code, raw_data) + + self.log_request_success(method, url, response.request.path_url, body, response.status_code, raw_data, duration) + + return response.status_code, response.headers, raw_data + + def close(self): + """ + Explicitly closes connections + """ + self.session.close()
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection/http_urllib3.py
Added
@@ -0,0 +1,140 @@ +import time +import urllib3 +from urllib3.exceptions import ReadTimeoutError, SSLError as UrllibSSLError +import warnings + +CA_CERTS = None + +try: + import certifi + CA_CERTS = certifi.where() +except ImportError: + pass + +from .base import Connection +from ..exceptions import ConnectionError, ImproperlyConfigured, ConnectionTimeout, SSLError +from ..compat import urlencode + +class Urllib3HttpConnection(Connection): + """ + Default connection class using the `urllib3` library and the http protocol. + + :arg host: hostname of the node (default: localhost) + :arg port: port to use (integer, default: 9200) + :arg url_prefix: optional url prefix for elasticsearch + :arg timeout: default timeout in seconds (float, default: 10) + :arg http_auth: optional http auth information as either ':' separated + string or a tuple + :arg use_ssl: use ssl for the connection if `True` + :arg verify_certs: whether to verify SSL certificates + :arg ca_certs: optional path to CA bundle. See + https://urllib3.readthedocs.io/en/latest/security.html#using-certifi-with-urllib3 + for instructions how to get default set + :arg client_cert: path to the file containing the private key and the + certificate, or cert only if using client_key + :arg client_key: path to the file containing the private key if using + separate cert and key files (client_cert will contain only the cert) + :arg ssl_version: version of the SSL protocol to use. Choices are: + SSLv23 (default) SSLv2 SSLv3 TLSv1 (see ``PROTOCOL_*`` constants in the + ``ssl`` module for exact options for your environment). + :arg ssl_assert_hostname: use hostname verification if not `False` + :arg ssl_assert_fingerprint: verify the supplied certificate fingerprint if not `None` + :arg maxsize: the number of connections which will be kept open to this + host. See https://urllib3.readthedocs.io/en/1.4/pools.html#api for more + information. + :arg headers: any custom http headers to be add to requests + """ + def __init__(self, host='localhost', port=9200, http_auth=None, + use_ssl=False, verify_certs=True, ca_certs=None, client_cert=None, + client_key=None, ssl_version=None, ssl_assert_hostname=None, + ssl_assert_fingerprint=None, maxsize=10, headers=None, **kwargs): + + super(Urllib3HttpConnection, self).__init__(host=host, port=port, use_ssl=use_ssl, **kwargs) + self.headers = urllib3.make_headers(keep_alive=True) + if http_auth is not None: + if isinstance(http_auth, (tuple, list)): + http_auth = ':'.join(http_auth) + self.headers.update(urllib3.make_headers(basic_auth=http_auth)) + + # update headers in lowercase to allow overriding of auth headers + if headers: + for k in headers: + self.headers[k.lower()] = headers[k] + + self.headers.setdefault('content-type', 'application/json') + ca_certs = CA_CERTS if ca_certs is None else ca_certs + pool_class = urllib3.HTTPConnectionPool + kw = {} + if use_ssl: + pool_class = urllib3.HTTPSConnectionPool + kw.update({ + 'ssl_version': ssl_version, + 'assert_hostname': ssl_assert_hostname, + 'assert_fingerprint': ssl_assert_fingerprint, + 'ca_certs': ca_certs, + 'cert_file': client_cert, + 'key_file': client_key, + }) + if verify_certs: + if not ca_certs: + raise ImproperlyConfigured("Root certificates are missing for certificate " + "validation. Either pass them in using the ca_certs parameter or " + "install certifi to use it automatically.") + kw.update({ + 'cert_reqs': 'CERT_REQUIRED', + }) + else: + warnings.warn( + 'Connecting to %s using SSL with verify_certs=False is insecure.' % host) + kw.update({ + 'cert_reqs': 'CERT_NONE', + }) + + self.pool = pool_class(host, port=port, timeout=self.timeout, maxsize=maxsize, **kw) + + def perform_request(self, method, url, params=None, body=None, timeout=None, ignore=()): + url = self.url_prefix + url + if params: + url = '%s?%s' % (url, urlencode(params)) + full_url = self.host + url + + start = time.time() + try: + kw = {} + if timeout: + kw['timeout'] = timeout + + # in python2 we need to make sure the url and method are not + # unicode. Otherwise the body will be decoded into unicode too and + # that will fail (#133, #201). + if not isinstance(url, str): + url = url.encode('utf-8') + if not isinstance(method, str): + method = method.encode('utf-8') + + response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw) + duration = time.time() - start + raw_data = response.data.decode('utf-8') + except Exception as e: + self.log_request_fail(method, full_url, url, body, time.time() - start, exception=e) + if isinstance(e, UrllibSSLError): + raise SSLError('N/A', str(e), e) + if isinstance(e, ReadTimeoutError): + raise ConnectionTimeout('TIMEOUT', str(e), e) + raise ConnectionError('N/A', str(e), e) + + # raise errors based on http status codes, let the client handle those if needed + if not (200 <= response.status < 300) and response.status not in ignore: + self.log_request_fail(method, full_url, url, body, duration, response.status, raw_data) + self._raise_error(response.status, raw_data) + + self.log_request_success(method, full_url, url, body, response.status, + raw_data, duration) + + return response.status, response.getheaders(), raw_data + + def close(self): + """ + Explicitly closes connection + """ + self.pool.close()
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection/pooling.py
Added
@@ -0,0 +1,33 @@ +try: + import queue +except ImportError: + import Queue as queue +from .base import Connection + + +class PoolingConnection(Connection): + """ + Base connection class for connections that use libraries without thread + safety and no capacity for connection pooling. To use this just implement a + ``_make_connection`` method that constructs a new connection and returns + it. + """ + def __init__(self, *args, **kwargs): + self._free_connections = queue.Queue() + super(PoolingConnection, self).__init__(*args, **kwargs) + + def _get_connection(self): + try: + return self._free_connections.get_nowait() + except queue.Empty: + return self._make_connection() + + def _release_connection(self, con): + self._free_connections.put(con) + + def close(self): + """ + Explicitly close connection + """ + pass +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/connection_pool.py
Added
@@ -0,0 +1,261 @@ +import time +import random +import logging +import threading + +try: + from Queue import PriorityQueue, Empty +except ImportError: + from queue import PriorityQueue, Empty + +from .exceptions import ImproperlyConfigured + +logger = logging.getLogger('elasticsearch') + +class ConnectionSelector(object): + """ + Simple class used to select a connection from a list of currently live + connection instances. In init time it is passed a dictionary containing all + the connections' options which it can then use during the selection + process. When the `select` method is called it is given a list of + *currently* live connections to choose from. + + The options dictionary is the one that has been passed to + :class:`~elasticsearch.Transport` as `hosts` param and the same that is + used to construct the Connection object itself. When the Connection was + created from information retrieved from the cluster via the sniffing + process it will be the dictionary returned by the `host_info_callback`. + + Example of where this would be useful is a zone-aware selector that would + only select connections from it's own zones and only fall back to other + connections where there would be none in it's zones. + """ + def __init__(self, opts): + """ + :arg opts: dictionary of connection instances and their options + """ + self.connection_opts = opts + + def select(self, connections): + """ + Select a connection from the given list. + + :arg connections: list of live connections to choose from + """ + pass + + +class RandomSelector(ConnectionSelector): + """ + Select a connection at random + """ + def select(self, connections): + return random.choice(connections) + + +class RoundRobinSelector(ConnectionSelector): + """ + Selector using round-robin. + """ + def __init__(self, opts): + super(RoundRobinSelector, self).__init__(opts) + self.data = threading.local() + + def select(self, connections): + self.data.rr = getattr(self.data, 'rr', -1) + 1 + self.data.rr %= len(connections) + return connections[self.data.rr] + +class ConnectionPool(object): + """ + Container holding the :class:`~elasticsearch.Connection` instances, + managing the selection process (via a + :class:`~elasticsearch.ConnectionSelector`) and dead connections. + + It's only interactions are with the :class:`~elasticsearch.Transport` class + that drives all the actions within `ConnectionPool`. + + Initially connections are stored on the class as a list and, along with the + connection options, get passed to the `ConnectionSelector` instance for + future reference. + + Upon each request the `Transport` will ask for a `Connection` via the + `get_connection` method. If the connection fails (it's `perform_request` + raises a `ConnectionError`) it will be marked as dead (via `mark_dead`) and + put on a timeout (if it fails N times in a row the timeout is exponentially + longer - the formula is `default_timeout * 2 ** (fail_count - 1)`). When + the timeout is over the connection will be resurrected and returned to the + live pool. A connection that has been previously marked as dead and + succeeds will be marked as live (its fail count will be deleted). + """ + def __init__(self, connections, dead_timeout=60, timeout_cutoff=5, + selector_class=RoundRobinSelector, randomize_hosts=True, **kwargs): + """ + :arg connections: list of tuples containing the + :class:`~elasticsearch.Connection` instance and it's options + :arg dead_timeout: number of seconds a connection should be retired for + after a failure, increases on consecutive failures + :arg timeout_cutoff: number of consecutive failures after which the + timeout doesn't increase + :arg selector_class: :class:`~elasticsearch.ConnectionSelector` + subclass to use if more than one connection is live + :arg randomize_hosts: shuffle the list of connections upon arrival to + avoid dog piling effect across processes + """ + if not connections: + raise ImproperlyConfigured("No defined connections, you need to " + "specify at least one host.") + self.connection_opts = connections + self.connections = [c for (c, opts) in connections] + # remember original connection list for resurrect(force=True) + self.orig_connections = tuple(self.connections) + # PriorityQueue for thread safety and ease of timeout management + self.dead = PriorityQueue(len(self.connections)) + self.dead_count = {} + + if randomize_hosts: + # randomize the connection list to avoid all clients hitting same node + # after startup/restart + random.shuffle(self.connections) + + # default timeout after which to try resurrecting a connection + self.dead_timeout = dead_timeout + self.timeout_cutoff = timeout_cutoff + + self.selector = selector_class(dict(connections)) + + def mark_dead(self, connection, now=None): + """ + Mark the connection as dead (failed). Remove it from the live pool and + put it on a timeout. + + :arg connection: the failed instance + """ + # allow inject for testing purposes + now = now if now else time.time() + try: + self.connections.remove(connection) + except ValueError: + # connection not alive or another thread marked it already, ignore + return + else: + dead_count = self.dead_count.get(connection, 0) + 1 + self.dead_count[connection] = dead_count + timeout = self.dead_timeout * 2 ** min(dead_count - 1, self.timeout_cutoff) + self.dead.put((now + timeout, connection)) + logger.warning( + 'Connection %r has failed for %i times in a row, putting on %i second timeout.', + connection, dead_count, timeout + ) + + def mark_live(self, connection): + """ + Mark connection as healthy after a resurrection. Resets the fail + counter for the connection. + + :arg connection: the connection to redeem + """ + try: + del self.dead_count[connection] + except KeyError: + # race condition, safe to ignore + pass + + def resurrect(self, force=False): + """ + Attempt to resurrect a connection from the dead pool. It will try to + locate one (not all) eligible (it's timeout is over) connection to + return to the live pool. Any resurrected connection is also returned. + + :arg force: resurrect a connection even if there is none eligible (used + when we have no live connections). If force is specified resurrect + always returns a connection. + + """ + # no dead connections + if self.dead.empty(): + # we are forced to return a connection, take one from the original + # list. This is to avoid a race condition where get_connection can + # see no live connections but when it calls resurrect self.dead is + # also empty. We assume that other threat has resurrected all + # available connections so we can safely return one at random. + if force: + return random.choice(self.orig_connections) + return + + try: + # retrieve a connection to check + timeout, connection = self.dead.get(block=False) + except Empty: + # other thread has been faster and the queue is now empty. If we + # are forced, return a connection at random again. + if force: + return random.choice(self.orig_connections) + return + + if not force and timeout > time.time(): + # return it back if not eligible and not forced + self.dead.put((timeout, connection)) + return
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/exceptions.py
Added
@@ -0,0 +1,109 @@ +__all__ = [ + 'ImproperlyConfigured', 'ElasticsearchException', 'SerializationError', + 'TransportError', 'NotFoundError', 'ConflictError', 'RequestError', 'ConnectionError', + 'SSLError', 'ConnectionTimeout', 'AuthenticationException', 'AuthorizationException' +] + +class ImproperlyConfigured(Exception): + """ + Exception raised when the config passed to the client is inconsistent or invalid. + """ + + +class ElasticsearchException(Exception): + """ + Base class for all exceptions raised by this package's operations (doesn't + apply to :class:`~elasticsearch.ImproperlyConfigured`). + """ + + +class SerializationError(ElasticsearchException): + """ + Data passed in failed to serialize properly in the ``Serializer`` being + used. + """ + + +class TransportError(ElasticsearchException): + """ + Exception raised when ES returns a non-OK (>=400) HTTP status code. Or when + an actual connection error happens; in that case the ``status_code`` will + be set to ``'N/A'``. + """ + @property + def status_code(self): + """ + The HTTP status code of the response that precipitated the error or + ``'N/A'`` if not applicable. + """ + return self.args[0] + + @property + def error(self): + """ A string error message. """ + return self.args[1] + + @property + def info(self): + """ Dict of returned error info from ES, where available. """ + return self.args[2] + + def __str__(self): + cause = '' + try: + if self.info: + cause = ', %r' % self.info['error']['root_cause'][0]['reason'] + except LookupError: + pass + return 'TransportError(%s, %r%s)' % (self.status_code, self.error, cause) + + +class ConnectionError(TransportError): + """ + Error raised when there was an exception while talking to ES. Original + exception from the underlying :class:`~elasticsearch.Connection` + implementation is available as ``.info.`` + """ + def __str__(self): + return 'ConnectionError(%s) caused by: %s(%s)' % ( + self.error, self.info.__class__.__name__, self.info) + + +class SSLError(ConnectionError): + """ Error raised when encountering SSL errors. """ + + +class ConnectionTimeout(ConnectionError): + """ A network timeout. Doesn't cause a node retry by default. """ + def __str__(self): + return 'ConnectionTimeout caused by - %s(%s)' % ( + self.info.__class__.__name__, self.info) + + +class NotFoundError(TransportError): + """ Exception representing a 404 status code. """ + + +class ConflictError(TransportError): + """ Exception representing a 409 status code. """ + + +class RequestError(TransportError): + """ Exception representing a 400 status code. """ + + +class AuthenticationException(TransportError): + """ Exception representing a 401 status code. """ + + +class AuthorizationException(TransportError): + """ Exception representing a 403 status code. """ + +# more generic mappings from status_code to python exceptions +HTTP_EXCEPTIONS = { + 400: RequestError, + 401: AuthenticationException, + 403: AuthorizationException, + 404: NotFoundError, + 409: ConflictError, +}
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/helpers
Added
+(directory)
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/helpers/__init__.py
Added
@@ -0,0 +1,458 @@ +from __future__ import unicode_literals + +import logging +from operator import methodcaller +import time + +from ..exceptions import ElasticsearchException, TransportError +from ..compat import map, string_types, Queue + + +logger = logging.getLogger('elasticsearch.helpers') + +class BulkIndexError(ElasticsearchException): + @property + def errors(self): + """ List of errors from execution of the last chunk. """ + return self.args[1] + + +class ScanError(ElasticsearchException): + def __init__(self, scroll_id, *args, **kwargs): + super(ScanError, self).__init__(*args, **kwargs) + self.scroll_id = scroll_id + +def expand_action(data): + """ + From one document or action definition passed in by the user extract the + action/data lines needed for elasticsearch's + :meth:`~elasticsearch.Elasticsearch.bulk` api. + """ + # when given a string, assume user wants to index raw json + if isinstance(data, string_types): + return '{"index":{}}', data + + # make sure we don't alter the action + data = data.copy() + op_type = data.pop('_op_type', 'index') + action = {op_type: {}} + for key in ('_index', '_parent', '_percolate', '_routing', '_timestamp', + '_type', '_version', '_version_type', '_id', + '_retry_on_conflict', 'pipeline'): + if key in data: + action[op_type][key] = data.pop(key) + + # no data payload for delete + if op_type == 'delete': + return action, None + + return action, data.get('_source', data) + +def _chunk_actions(actions, chunk_size, max_chunk_bytes, serializer): + """ + Split actions into chunks by number or size, serialize them into strings in + the process. + """ + bulk_actions, bulk_data = [], [] + size, action_count = 0, 0 + for action, data in actions: + raw_data, raw_action = data, action + action = serializer.dumps(action) + cur_size = len(action) + 1 + + if data is not None: + data = serializer.dumps(data) + cur_size += len(data) + 1 + + # full chunk, send it and start a new one + if bulk_actions and (size + cur_size > max_chunk_bytes or action_count == chunk_size): + yield bulk_data, bulk_actions + bulk_actions, bulk_data = [], [] + size, action_count = 0, 0 + + bulk_actions.append(action) + if data is not None: + bulk_actions.append(data) + bulk_data.append((raw_action, raw_data)) + else: + bulk_data.append((raw_action, )) + + size += cur_size + action_count += 1 + + if bulk_actions: + yield bulk_data, bulk_actions + +def _process_bulk_chunk(client, bulk_actions, bulk_data, raise_on_exception=True, raise_on_error=True, **kwargs): + """ + Send a bulk request to elasticsearch and process the output. + """ + # if raise on error is set, we need to collect errors per chunk before raising them + errors = [] + + try: + # send the actual request + resp = client.bulk('\n'.join(bulk_actions) + '\n', **kwargs) + except TransportError as e: + # default behavior - just propagate exception + if raise_on_exception: + raise e + + # if we are not propagating, mark all actions in current chunk as failed + err_message = str(e) + exc_errors = [] + + for data in bulk_data: + # collect all the information about failed actions + op_type, action = data[0].copy().popitem() + info = {"error": err_message, "status": e.status_code, "exception": e} + if op_type != 'delete': + info['data'] = data[1] + info.update(action) + exc_errors.append({op_type: info}) + + # emulate standard behavior for failed actions + if raise_on_error: + raise BulkIndexError('%i document(s) failed to index.' % len(exc_errors), exc_errors) + else: + for err in exc_errors: + yield False, err + return + + # go through request-reponse pairs and detect failures + for data, (op_type, item) in zip(bulk_data, map(methodcaller('popitem'), resp['items'])): + ok = 200 <= item.get('status', 500) < 300 + if not ok and raise_on_error: + # include original document source + if len(data) > 1: + item['data'] = data[1] + errors.append({op_type: item}) + + if ok or not errors: + # if we are not just recording all errors to be able to raise + # them all at once, yield items individually + yield ok, {op_type: item} + + if errors: + raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors) + +def streaming_bulk(client, actions, chunk_size=500, max_chunk_bytes=100 * 1024 * 1024, + raise_on_error=True, expand_action_callback=expand_action, + raise_on_exception=True, max_retries=0, initial_backoff=2, + max_backoff=600, yield_ok=True, **kwargs): + + """ + Streaming bulk consumes actions from the iterable passed in and yields + results per action. For non-streaming usecases use + :func:`~elasticsearch.helpers.bulk` which is a wrapper around streaming + bulk that returns summary information about the bulk operation once the + entire input is consumed and sent. + + If you specify ``max_retries`` it will also retry any documents that were + rejected with a ``429`` status code. To do this it will wait (**by calling + time.sleep which will block**) for ``initial_backoff`` seconds and then, + every subsequent rejection for the same chunk, for double the time every + time up to ``max_backoff`` seconds. + + :arg client: instance of :class:`~elasticsearch.Elasticsearch` to use + :arg actions: iterable containing the actions to be executed + :arg chunk_size: number of docs in one chunk sent to es (default: 500) + :arg max_chunk_bytes: the maximum size of the request in bytes (default: 100MB) + :arg raise_on_error: raise ``BulkIndexError`` containing errors (as `.errors`) + from the execution of the last chunk when some occur. By default we raise. + :arg raise_on_exception: if ``False`` then don't propagate exceptions from + call to ``bulk`` and just report the items that failed as failed. + :arg expand_action_callback: callback executed on each action passed in, + should return a tuple containing the action line and the data line + (`None` if data line should be omitted). + :arg max_retries: maximum number of times a document will be retried when + ``429`` is received, set to 0 (default) for no retires on ``429`` + :arg initial_backoff: number of seconds we should wait before the first + retry. Any subsequent retries will be powers of ``inittial_backoff * + 2**retry_number`` + :arg max_backoff: maximum number of seconds a retry will wait + :arg yield_ok: if set to False will skip successful documents in the output + """ + actions = map(expand_action_callback, actions) + + for bulk_data, bulk_actions in _chunk_actions(actions, chunk_size, + max_chunk_bytes, + client.transport.serializer): + + for attempt in range(max_retries + 1): + to_retry, to_retry_data = [], [] + if attempt: + time.sleep(min(max_backoff, initial_backoff * 2**(attempt-1))) + + try: + for data, (ok, info) in zip( + bulk_data, + _process_bulk_chunk(client, bulk_actions, bulk_data, + raise_on_exception, + raise_on_error, **kwargs) + ): + + if not ok: + action, info = info.popitem() + # retry if retries enabled, we get 429, and we are not + # in the last attempt + if max_retries \
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/helpers/test.py
Added
@@ -0,0 +1,60 @@ +import time +import os +try: + # python 2.6 + from unittest2 import TestCase, SkipTest +except ImportError: + from unittest import TestCase, SkipTest + +from elasticsearch5 import Elasticsearch +from elasticsearch5.exceptions import ConnectionError + +def get_test_client(nowait=False, **kwargs): + # construct kwargs from the environment + kw = {'timeout': 30} + if 'TEST_ES_CONNECTION' in os.environ: + from elasticsearch5 import connection + kw['connection_class'] = getattr(connection, os.environ['TEST_ES_CONNECTION']) + + kw.update(kwargs) + client = Elasticsearch([os.environ.get('TEST_ES_SERVER', {})], **kw) + + # wait for yellow status + for _ in range(1 if nowait else 100): + try: + client.cluster.health(wait_for_status='yellow') + return client + except ConnectionError: + time.sleep(.1) + else: + # timeout + raise SkipTest("Elasticsearch failed to start.") + +def _get_version(version_string): + if '.' not in version_string: + return () + version = version_string.strip().split('.') + return tuple(int(v) if v.isdigit() else 999 for v in version) + +class ElasticsearchTestCase(TestCase): + @staticmethod + def _get_client(): + return get_test_client() + + @classmethod + def setUpClass(cls): + super(ElasticsearchTestCase, cls).setUpClass() + cls.client = cls._get_client() + + def tearDown(self): + super(ElasticsearchTestCase, self).tearDown() + self.client.indices.delete(index='*', ignore=404) + self.client.indices.delete_template(name='*', ignore=404) + + @property + def es_version(self): + if not hasattr(self, '_es_version'): + version_string = self.client.info()['version']['number'] + self._es_version = _get_version(version_string) + return self._es_version +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/serializer.py
Added
@@ -0,0 +1,77 @@ +try: + import simplejson as json +except ImportError: + import json +import uuid +from datetime import date, datetime +from decimal import Decimal + +from .exceptions import SerializationError, ImproperlyConfigured +from .compat import string_types + +class TextSerializer(object): + mimetype = 'text/plain' + + def loads(self, s): + return s + + def dumps(self, data): + if isinstance(data, string_types): + return data + + raise SerializationError('Cannot serialize %r into text.' % data) + +class JSONSerializer(object): + mimetype = 'application/json' + + def default(self, data): + if isinstance(data, (date, datetime)): + return data.isoformat() + elif isinstance(data, Decimal): + return float(data) + elif isinstance(data, uuid.UUID): + return str(data) + raise TypeError("Unable to serialize %r (type: %s)" % (data, type(data))) + + def loads(self, s): + try: + return json.loads(s) + except (ValueError, TypeError) as e: + raise SerializationError(s, e) + + def dumps(self, data): + # don't serialize strings + if isinstance(data, string_types): + return data + + try: + return json.dumps(data, default=self.default, ensure_ascii=False) + except (ValueError, TypeError) as e: + raise SerializationError(data, e) + +DEFAULT_SERIALIZERS = { + JSONSerializer.mimetype: JSONSerializer(), + TextSerializer.mimetype: TextSerializer(), +} + +class Deserializer(object): + def __init__(self, serializers, default_mimetype='application/json'): + try: + self.default = serializers[default_mimetype] + except KeyError: + raise ImproperlyConfigured('Cannot find default serializer (%s)' % default_mimetype) + self.serializers = serializers + + def loads(self, s, mimetype=None): + if not mimetype: + deserializer = self.default + else: + # split out charset + mimetype = mimetype.split(';', 1)[0] + try: + deserializer = self.serializers[mimetype] + except KeyError: + raise SerializationError('Unknown mimetype, unable to deserialize: %s' % mimetype) + + return deserializer.loads(s) +
View file
elasticsearch5-5.5.3.tar.gz/elasticsearch5/transport.py
Added
@@ -0,0 +1,349 @@ +import time +from itertools import chain + +from .connection import Urllib3HttpConnection +from .connection_pool import ConnectionPool, DummyConnectionPool +from .serializer import JSONSerializer, Deserializer, DEFAULT_SERIALIZERS +from .exceptions import ConnectionError, TransportError, SerializationError, \ + ConnectionTimeout, ImproperlyConfigured + + +def get_host_info(node_info, host): + """ + Simple callback that takes the node info from `/_cluster/nodes` and a + parsed connection information and return the connection information. If + `None` is returned this node will be skipped. + + Useful for filtering nodes (by proximity for example) or if additional + information needs to be provided for the :class:`~elasticsearch.Connection` + class. By default master only nodes are filtered out since they shouldn't + typically be used for API operations. + + :arg node_info: node information from `/_cluster/nodes` + :arg host: connection information (host, port) extracted from the node info + """ + # ignore master only nodes + if node_info.get('roles', []) == ['master']: + return None + return host + +class Transport(object): + """ + Encapsulation of transport-related to logic. Handles instantiation of the + individual connections as well as creating a connection pool to hold them. + + Main interface is the `perform_request` method. + """ + def __init__(self, hosts, connection_class=Urllib3HttpConnection, + connection_pool_class=ConnectionPool, host_info_callback=get_host_info, + sniff_on_start=False, sniffer_timeout=None, sniff_timeout=.1, + sniff_on_connection_fail=False, serializer=JSONSerializer(), serializers=None, + default_mimetype='application/json', max_retries=3, retry_on_status=(502, 503, 504, ), + retry_on_timeout=False, send_get_body_as='GET', **kwargs): + """ + :arg hosts: list of dictionaries, each containing keyword arguments to + create a `connection_class` instance + :arg connection_class: subclass of :class:`~elasticsearch.Connection` to use + :arg connection_pool_class: subclass of :class:`~elasticsearch.ConnectionPool` to use + :arg host_info_callback: callback responsible for taking the node information from + `/_cluser/nodes`, along with already extracted information, and + producing a list of arguments (same as `hosts` parameter) + :arg sniff_on_start: flag indicating whether to obtain a list of nodes + from the cluser at startup time + :arg sniffer_timeout: number of seconds between automatic sniffs + :arg sniff_on_connection_fail: flag controlling if connection failure triggers a sniff + :arg sniff_timeout: timeout used for the sniff request - it should be a + fast api call and we are talking potentially to more nodes so we want + to fail quickly. Not used during initial sniffing (if + ``sniff_on_start`` is on) when the connection still isn't + initialized. + :arg serializer: serializer instance + :arg serializers: optional dict of serializer instances that will be + used for deserializing data coming from the server. (key is the mimetype) + :arg default_mimetype: when no mimetype is specified by the server + response assume this mimetype, defaults to `'application/json'` + :arg max_retries: maximum number of retries before an exception is propagated + :arg retry_on_status: set of HTTP status codes on which we should retry + on a different node. defaults to ``(502, 503, 504)`` + :arg retry_on_timeout: should timeout trigger a retry on different + node? (default `False`) + :arg send_get_body_as: for GET requests with body this option allows + you to specify an alternate way of execution for environments that + don't support passing bodies with GET requests. If you set this to + 'POST' a POST method will be used instead, if to 'source' then the body + will be serialized and passed as a query parameter `source`. + + Any extra keyword arguments will be passed to the `connection_class` + when creating and instance unless overridden by that connection's + options provided as part of the hosts parameter. + """ + + # serialization config + _serializers = DEFAULT_SERIALIZERS.copy() + # if a serializer has been specified, use it for deserialization as well + _serializers[serializer.mimetype] = serializer + # if custom serializers map has been supplied, override the defaults with it + if serializers: + _serializers.update(serializers) + # create a deserializer with our config + self.deserializer = Deserializer(_serializers, default_mimetype) + + self.max_retries = max_retries + self.retry_on_timeout = retry_on_timeout + self.retry_on_status = retry_on_status + self.send_get_body_as = send_get_body_as + + # data serializer + self.serializer = serializer + + # store all strategies... + self.connection_pool_class = connection_pool_class + self.connection_class = connection_class + + # ...save kwargs to be passed to the connections + self.kwargs = kwargs + self.hosts = hosts + + # ...and instantiate them + self.set_connections(hosts) + # retain the original connection instances for sniffing + self.seed_connections = self.connection_pool.connections[:] + + # sniffing data + self.sniffer_timeout = sniffer_timeout + self.sniff_on_connection_fail = sniff_on_connection_fail + self.last_sniff = time.time() + self.sniff_timeout = sniff_timeout + + # callback to construct host dict from data in /_cluster/nodes + self.host_info_callback = host_info_callback + + if sniff_on_start: + self.sniff_hosts(True) + + def add_connection(self, host): + """ + Create a new :class:`~elasticsearch.Connection` instance and add it to the pool. + + :arg host: kwargs that will be used to create the instance + """ + self.hosts.append(host) + self.set_connections(self.hosts) + + def set_connections(self, hosts): + """ + Instantiate all the connections and crate new connection pool to hold + them. Tries to identify unchanged hosts and re-use existing + :class:`~elasticsearch.Connection` instances. + + :arg hosts: same as `__init__` + """ + # construct the connections + def _create_connection(host): + # if this is not the initial setup look at the existing connection + # options and identify connections that haven't changed and can be + # kept around. + if hasattr(self, 'connection_pool'): + for (connection, old_host) in self.connection_pool.connection_opts: + if old_host == host: + return connection + + # previously unseen params, create new connection + kwargs = self.kwargs.copy() + kwargs.update(host) + return self.connection_class(**kwargs) + connections = map(_create_connection, hosts) + + connections = list(zip(connections, hosts)) + if len(connections) == 1: + self.connection_pool = DummyConnectionPool(connections) + else: + # pass the hosts dicts to the connection pool to optionally extract parameters from + self.connection_pool = self.connection_pool_class(connections, **self.kwargs) + + def get_connection(self): + """ + Retreive a :class:`~elasticsearch.Connection` instance from the + :class:`~elasticsearch.ConnectionPool` instance. + """ + if self.sniffer_timeout: + if time.time() >= self.last_sniff + self.sniffer_timeout: + self.sniff_hosts() + return self.connection_pool.get_connection() + + def _get_sniff_data(self, initial=False): + """ + Perform the request to get sniffins information. Returns a list of + dictionaries (one per node) containing all the information from the + cluster. + + It also sets the last_sniff attribute in case of a successful attempt. + + In rare cases it might be possible to override this method in your + custom Transport class to serve data from alternative source like + configuration management. + """ + previous_sniff = self.last_sniff + + try: + # reset last_sniff timestamp + self.last_sniff = time.time() + # go through all current connections as well as the + # seed_connections for good measure + for c in chain(self.connection_pool.connections, self.seed_connections): + try: + # use small timeout for the sniffing request, should be a fast api call + _, headers, node_info = c.perform_request( + 'GET', '/_nodes/_all/http', + timeout=self.sniff_timeout if not initial else None) + node_info = self.deserializer.loads(node_info, headers.get('content-type'))
View file
elasticsearch-1.0.0.tar.gz/setup.cfg -> elasticsearch5-5.5.3.tar.gz/setup.cfg
Changed
@@ -6,8 +6,10 @@ [wheel] universal = 1 +[bdist_rpm] +requires = python python-urllib3 + [egg_info] tag_build = tag_date = 0 -tag_svn_revision = 0
View file
elasticsearch-1.0.0.tar.gz/setup.py -> elasticsearch5-5.5.3.tar.gz/setup.py
Changed
@@ -2,21 +2,20 @@ from os.path import join, dirname from setuptools import setup, find_packages import sys -import os -VERSION = (1, 0, 0) +VERSION = (5, 5, 3) __version__ = VERSION __versionstr__ = '.'.join(map(str, VERSION)) -f = open(join(dirname(__file__), 'README.rst')) +f = open(join(dirname(__file__), 'README')) long_description = f.read().strip() f.close() install_requires = [ - 'urllib3>=1.5, <2.0', + 'urllib3>=1.21.1', ] tests_require = [ - 'requests>=1.0.0, <3.0.0', + 'requests>=2.0.0, <3.0.0', 'nose', 'coverage', 'mock', @@ -26,35 +25,36 @@ # use external unittest for 2.6 if sys.version_info[:2] == (2, 6): - tests_require.append('unittest2') - -if sys.version_info[0] == 2: - # only require thrift if we are going to use it - if os.environ.get('TEST_ES_CONNECTION', None) == 'ThriftConnection': - tests_require.append('thrift==0.9.1') - tests_require.append('pylibmc==1.2.3') + install_requires.append('unittest2') setup( - name = 'elasticsearch', + name = 'elasticsearch5', description = "Python client for Elasticsearch", license="Apache License, Version 2.0", - url = "https://github.com/elasticsearch/elasticsearch-py", + url = "https://github.com/elastic/elasticsearch-py", long_description = long_description, version = __versionstr__, - author = "Honza Král", - author_email = "honza.kral@gmail.com", + author = "Honza Král, Nick Lang", + author_email = "honza.kral@gmail.com, nick@nicklang.com", packages=find_packages( where='.', exclude=('test_elasticsearch*', ) ), classifiers = [ - "Development Status :: 4 - Beta", + "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: Apache Software License", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python", "Programming Language :: Python :: 2", + "Programming Language :: Python :: 2.6", + "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.2", + "Programming Language :: Python :: 3.3", + "Programming Language :: Python :: 3.4", + "Programming Language :: Python :: 3.5", + "Programming Language :: Python :: 3.6", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", ], @@ -62,4 +62,8 @@ test_suite='test_elasticsearch.run_tests.run_all', tests_require=tests_require, + + extras_require={ + 'develop': tests_require + ["sphinx", "sphinx_rtd_theme"] + }, )
View file
elasticsearch-1.0.0.tar.gz/tox.ini -> elasticsearch5-5.5.3.tar.gz/tox.ini
Changed
@@ -1,5 +1,5 @@ [tox] -envlist = pypy,py26,py27,py33 +envlist = pypy,py26,py27,py33,py34,py35,py36 [testenv] whitelist_externals = git setenv =
Locations
Projects
Search
Status Monitor
Help
Open Build Service
OBS Manuals
API Documentation
OBS Portal
Reporting a Bug
Contact
Mailing List
Forums
Chat (IRC)
Twitter
Open Build Service (OBS)
is an
openSUSE project
.