Lupyne is a search engine based on PyLucene, the Python extension for accessing Java Lucene. Lucene is a relatively low-level toolkit, and PyLucene wraps it through automatic code generation. So although Java idioms are translated to Python idioms where possible, the resulting interface is far from Pythonic. See ./docs/examples.ipynb for comparisons with the Lucene API.
Lupyne also provides GraphQL and RESTful search services, based on Starlette. Note Solr and Elasticsearch are popular options for Lucene-based search, if no further (Python) customization is needed. So while the services are suitable for production usage, their primary motivation is to be an extensible example.
Not having to initially choose between an embedded library and a server not only provides greater flexibility, it can provide better performance, e.g., batch indexing offline and remote searching live. Additionally only lightweight wrappers with extended behavior are used wherever possible, so falling back to using PyLucene directly is always an option, but should never be necessary for performance.
Usage
PyLucene requires initializing the VM.
import lucene
lucene.initVM()
Indexes are accessed through an IndexSearcher (read-only), IndexWriter, or the combined Indexer.
from lupyne import engine
searcher = engine.IndexSearcher('index/path')
hits = searcher.search('text:query')
See ./lupyne/services/README.md for services usage.
Installation
% pip install lupyne[graphql,rest]
PyLucene is not pip installable.
- Install instructions
- Docker image:
docker pull coady/pylucene - Homebrew formula:
brew install coady/tap/pylucene
Dependencies
- PyLucene >=8
- strawberry-graphql >=0.84.4 (if graphql option)
- fastapi (if rest option)
Tests
100% branch coverage.
% pytest [--cov]
Changes
dev
- PyLucene >=8.6 required
- PyLucene 8.11 supported
- CherryPy server removed
2.5
- Python >=3.7 required
- PyLucene 8.6 supported
- CherryPy server deprecated
2.4
- PyLucene >=8 required
Hit.keysrenamed toHit.sortkeys
2.3
- PyLucene >=7.7 required
- PyLucene 8 supported
2.2
- PyLucene 7.6 supported
2.1
- PyLucene >=7 required
2.0
- PyLucene >=6 required
- Python 3 support
- client moved to external package
1.9
- Python 2.6 dropped
- PyLucene 4.8 and 4.9 dropped
- IndexWriter implements context manager
- Server DocValues updated via patch method
- Spatial tile search optimized
1.8
- PyLucene 4.10 supported
- PyLucene 4.6 and 4.7 dropped
- Comparator iteration optimized
- Support for string based FieldCacheRangeFilters
How can I assign BM25 as scoring function? and should I give different setting like tokenization when searching different languages ?
Sorry for taking your time! Thanks !
33 Nov 21, 2022
4 Jun 29, 2021
5 Sep 24, 2021
1 Apr 4, 2022
1 Jun 17, 2022
2 Sep 12, 2022
155 Dec 30, 2022
3 Oct 28, 2022
0 Oct 6, 2022
4 Mar 8, 2022
3.4k Jan 4, 2023
197 Dec 29, 2022
17k Dec 31, 2022
152 Oct 29, 2022
119 Jan 6, 2023
49 Oct 30, 2022
10 Jun 20, 2022
1.1k Jan 3, 2023
394 Dec 12, 2022