Jina allows you to build deep learning-powered search-as-a-service in just minutes

Overview

Jina banner

Cloud-Native Neural Search[?] Framework for Any Kind of Data

Python 3.7 3.8 3.9 Docker Image Version (latest semver) codecov

Jina allows you to build deep learning-powered search-as-a-service in just minutes.

๐ŸŒŒ All data type - Large-scale indexing and querying of any kind of unstructured data: video, image, long/short text, music, source code, PDF, etc.

๐ŸŒฉ๏ธ Fast & cloud-native - Distributed architecture from day one. Scalable & cloud-native by design: enjoy containerizing, distributing, streaming, paralleling, sharding, async scheduling with REST/gRPC/WebSocket.

โฑ๏ธ Save time - The design pattern of neural search systems, from zero to a production-ready system in minutes.

๐Ÿฑ Own your stack - Keep an end-to-end stack ownership of your solution, avoid integration pitfalls with fragmented, multi-vendor, generic legacy tools.

Run Quick Demo

Install

2.0 is in pre-release, add --pre to install it.

$ pip install --pre jina
$ jina -v
2.0.0rcN

via Docker

$ docker run jinaai/jina:master -v
2.0.0rcN
๐Ÿ“ฆ More installation options

x86/64,arm64,v6,v7,Apple M1
On Linux/macOS & Python 3.7/3.8/3.9 Docker Users
Standard pip install --pre jina docker run jinaai/jina:master
Daemon pip install --pre "jina[daemon]" docker run --network=host jinaai/jina:master-daemon
With Extras pip install --pre "jina[devel]" docker run jinaai/jina:master-devel

Version identifiers are explained here. Jina can run on Windows Subsystem for Linux. We welcome the community to help us with native Windows support.

Get Started

Document, Executor, and Flow are the three fundamental concepts in Jina.

Copy-paste the minimum example below and run it:

๐Ÿ’ก Preliminaries: character embedding, pooling, Euclidean distance

Get started system diagram

import numpy as np
from jina import Document, DocumentArray, Executor, Flow, requests

class CharEmbed(Executor):  # a simple character embedding with mean-pooling
    offset = 32  # letter `a`
    dim = 127 - offset + 1  # last pos reserved for `UNK`
    char_embd = np.eye(dim) * 1  # one-hot embedding for all chars

    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        for d in docs:
            r_emb = [ord(c) - self.offset if self.offset <= ord(c) <= 127 else (self.dim - 1) for c in d.text]
            d.embedding = self.char_embd[r_emb, :].mean(axis=0)  # average pooling

class Indexer(Executor):
    _docs = DocumentArray()  # for storing all documents in memory

    @requests(on='/index')
    def foo(self, docs: DocumentArray, **kwargs):
        self._docs.extend(docs)  # extend stored `docs`

    @requests(on='/search')
    def bar(self, docs: DocumentArray, **kwargs):
        q = np.stack(docs.get_attributes('embedding'))  # get all embeddings from query docs
        d = np.stack(self._docs.get_attributes('embedding'))  # get all embeddings from stored docs
        euclidean_dist = np.linalg.norm(q[:, None, :] - d[None, :, :], axis=-1)  # pairwise euclidean distance
        for dist, query in zip(euclidean_dist, docs):  # add & sort match
            query.matches = [Document(self._docs[int(idx)], copy=True, scores={'euclid': d}) for idx, d in enumerate(dist)]
            query.matches.sort(key=lambda m: m.scores['euclid'].value)  # sort matches by their values

f = Flow(port_expose=12345).add(uses=CharEmbed, parallel=2).add(uses=Indexer)  # build a Flow, with 2 parallel CharEmbed, tho unnecessary
with f:
    f.post('/index', (Document(text=t.strip()) for t in open(__file__) if t.strip()))  # index all lines of this file
    f.block()  # block for listening request

Keep the above running and start a simple client:

from jina import Client, Document
from jina.types.request import Response


def print_matches(resp: Response):  # the callback function invoked when task is done
    for idx, d in enumerate(resp.docs[0].matches[:3]):  # print top-3 matches
        print(f'[{idx}]{d.scores["euclid"].value:2f}: "{d.text}"')


c = Client(host='localhost', port_expose=12345)  # connect to localhost:12345
c.post('/search', Document(text='request(on=something)'), on_done=print_matches)

It finds the lines most similar to "request(on=something)" from the server code snippet and prints the following:

         Client@1608[S]:connected to the gateway at localhost:12345!
[0]0.168526: "@requests(on='/index')"
[1]0.181676: "@requests(on='/search')"
[2]0.192049: "query.matches = [Document(self._docs[int(idx)], copy=True, score=d) for idx, d in enumerate(dist)]"

๐Ÿ˜” Doesn't work? Our bad! Please report it here.

Read Tutorials

Support

Join Us

Jina is backed by Jina AI. We are actively hiring full-stack developers, solution engineers to build the next neural search ecosystem in open source.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

All Contributors

Comments
  • Text-by-Audio

    Text-by-Audio

    Hi,

    I want to retrieve text by searching for an audio using AudioClip model.

    First, I created indexing of text (car-horn, coughing, alarm-clock, thunderstorm etc) and using AudioClipTextEncoder for embedding.

    After that I am searching for an audio where i am using AudioClipEncoder for embedding.

    For both text and audio indexing simple-indexer is used.

    While searching for an audio i created chunks using segmenter and using MyRanker for ranking scores where i modified some script.

    The output is: input_audio: AudioCLIP/demo/audio/thunder_3-144891-B-19.wav, len-of-chunks:1 +- id: e5eceb26737311ec83fd2fdc0fc83b8e score: 0.931110, cat input_audio: AudioCLIP/demo/audio/coughing_1-58792-A-24.wav, len-of-chunks:1 +- id: e5ecf968737311ec83fd2fdc0fc83b8e score: 0.923869, cat input_audio: AudioCLIP/demo/audio/cat_3-95694-A-5.wav, len-of-chunks:1 +- id: e5ed00de737311ec83fd2fdc0fc83b8e score: 0.845475, cat input_audio: AudioCLIP/demo/audio/car_horn_1-24074-A-43.wav, len-of-chunks:1 +- id: e5ed0778737311ec83fd2fdc0fc83b8e score: 0.928283, cat input_audio: AudioCLIP/demo/audio/alarm_clock_3-120526-B-37.wav, len-of-chunks:1 +- id: e5ed0dcc737311ec83fd2fdc0fc83b8e score: 0.919222, alarm_clock

    As above for 5 inputs only 2 outputs are correct. Please let me know why is that so ? Since AudioClipEncoder is giving correct outputs for all audios.

    opened by snaaz21 55
  • [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    Describe your proposal/problem

    Dear Jina Team,

    Recently, I tried nlp-simple example and replaced TransformerTorchEncoder with TextPaddlehubEncoder. However, I got the following error.

    Traceback (most recent call last):
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 73, in _load_executor
        self._executor = BaseExecutor.load_config(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 531, in load_config
        return JAML.load(tag_yml, substitute=False)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 89, in load
        r = yaml.load(stream, Loader=JinaLoader)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
        return loader.get_single_data()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
        return self.construct_document(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
        data = self.construct_object(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
        data = constructor(self, node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 422, in _from_yaml
        return get_parser(cls, version=data.get('version', None)).parse(cls, data)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/parsers/executor/legacy.py", line 130, in parse
        obj = cls(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 82, in __call__
        getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 174, in _post_init_wrapper
        self.post_init()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/hub/encoders/nlp/TextPaddlehubEncoder/__init__.py", line 38, in post_init
        self.model = hub.Module(name=self.model_name)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 171, in __new__
        module = cls.init_with_name(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 263, in init_with_name
        user_module_cls = manager.install(name=name, version=version, source=source, update=update, branch=branch)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 188, in install
        return self._install_from_name(name, version)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 263, in _install_from_name
        return self._install_from_url(item['url'])
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 256, in _install_from_url
        return self._install_from_archive(file)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 361, in _install_from_archive
        return self._install_from_directory(directory)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 345, in _install_from_directory
        hub_module_cls = HubModule.load(self._get_normalized_path(module_info.name))
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 219, in load
        paddle.set_device(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/device.py", line 166, in set_device
        framework._set_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 317, in _set_expected_place
        _set_dygraph_tracer_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 311, in _set_dygraph_tracer_expected_place
        _dygraph_tracer_._expected_place = place
    OSError: (External)  Cuda error(3), initialization error.
      [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:229)
    

    It seems to be related with https://github.com/PaddlePaddle/Paddle/issues/25185.

    However, I have no idea how to fix this. Could you help me to figure it out? Thank you very much!


    Environment

    paddlehub==2.0.0 paddlenlp==2.0.1 paddlepaddle-gpu==2.0.2.post90 jina==1.3.0

    Screenshots

    image

    area/core type/bug 
    opened by ryh95 47
  • feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    Goals:

    • resolves #5155
    • [x] Integrate OpenTelemetry API and SDK.
    • ~[ ] Provide environment variable configurations to enable tracking when required. Use console exporter for now.~
    • [x] Trace gRPC requests within the Flow.
    • [x] Add helpers for creating traces on request methods with default span attributes.
    • ~[ ] Convert send_health_check_sync or is_ready method to async to prevent the grpc aio interceptor from throwing and capturing an exception.~
    • [x] Extract tracing context from the server and make it available for the Executor methods in the kwargs list or arguments.
    • [ ] check and update documentation. See guide and ask the team.

    Sample Usage

    Flow

    jtype: Flow
    version: '1'
    with:
      protocol: grpc
      port: 54321
      tracing: true
      traces_exporter_host: '0.0.0.0'
      traces_exporter_port: 4317
    executors:
      - uses: executor1/config.yml
        name: toyExecutor
      - uses: executor2/config.yml
        name: toyExecutor2
    

    Executor

    import functools
    
    from jina import DocumentArray, Executor, requests
    from opentelemetry.context.context import Context
    from opentelemetry.semconv.trace import SpanAttributes
    from opentelemetry.trace import Status, StatusCode
    
    
    class MyExecutor(Executor):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
        @requests
        def foo(self, docs: DocumentArray, tracing_context: Context, **kwargs):
            with self.tracer.start_span("foo", context=tracing_context) as span:
                try:
                    span.set_attribute("len_added_docs", len(docs))
                    span.set_attribute(SpanAttributes.RPC_METHOD, functools.__name__)
    
                    docs[0].text = 'hello, world!'
                    docs[1].text = 'goodbye, world!'
                except Exception as ex:
                    span.set_status(Status(StatusCode.ERROR))
                    span.record_exception(ex)
                finally:
                    span.set_status(Status(StatusCode.OK))
    

    Client

    from jina import Client, DocumentArray
    import time
    
    if __name__ == '__main__':
        c = Client(
            host='grpc://0.0.0.0:54321',
            tracing=True,
            traces_exporter_host='0.0.0.0',
            traces_exporter_port=4317,
        )
    
        da = c.post('/', DocumentArray.empty(4))
        print(da.texts)
    
        time.sleep(3)
    
    

    Collecting Data

    Please check the docker-compose.yml and otel-collector-config.yml under the folder tests/integration/instrumentation for running the OpenTelemetry collector and Jaeger UI locally.

    size/L area/core size/S area/testing size/XL area/docs component/resource area/setup component/client area/cli 
    opened by girishc13 45
  • No documents are found with BinaryPbIndexer in 1.1.2

    No documents are found with BinaryPbIndexer in 1.1.2

    Versions 1.0.11: everything works as expected 1.1.2: BinaryPbIndexer doesn't find any documents

    Detailed description He Jina team, when I upgrade to 1.1.2 and reindex my dataset, the BinaryPbIndexer no longer works. It doesn't raise any errors, but I just can't find any documents. Just as a remark, I use custom ids which always consist of 33 characters. Where there any changes for the BinaryPbIndexer, the KVIndexDriver or the KVSearchDriver?

    area/core priority/critical type/bug status/done 
    opened by janandreschweiger 39
  • Permanently delete documents from storage

    Permanently delete documents from storage

    Hey Jina team!

    As described here documents persist the deletion and are only marked as inaccessible.

    My question: Is there a possibility to delete documents from storage without reindexing everything again? Deletion performance isn't an issue for us, but reindexing everything is very inconvenient in our use case.

    Thanks! I really appreciate your kind support.

    Sub-tasks

    • [x] Delete from NpIndexer. @ #2046
    • [x] Delete from BinaryPbIndexer. @ https://github.com/jina-ai/jina/pull/2102
    area/core priority/important-soon type/feature-request status/done 
    opened by ace-kay-law-neo 39
  • File seems to be closed when trying to index

    File seems to be closed when trying to index

    Yes @JoanFM, using the prefetch argument in the flow-config solved the original problem. I tried to pass the batch_size and the request_size like you, but I still get the same error. I have some additional logs, which might indicate what causes the problem. As I said, the first few batches are indexed successfully, but at some point I get this error:

            encoder@63[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            encoder@63[I]:#sent: 3 #recv: 4 sent_size: 13.8 MB recv_size: 1.2 MB
          chunk_idx@66[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            encoder@63[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
       NumpyIndexer@66[S]:artifacts of this executor (wrap_idx) is persisted to /var/huforce/fs/workspace/wrap_idx-0/wrap_idx.bin
    BinaryPbIndexer@66[S]:artifacts of this executor (chunk_idx) is persisted to /var/huforce/fs/workspace/chunk_idx-0/chunk_idx.bin
          chunk_idx@66[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
          chunk_idx@66[I]:#sent: 3 #recv: 4 sent_size: 1.9 MB recv_size: 15.6 MB
            doc_idx@67[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธdoc_idx/ZEDRuntimeโ–ธโš
    DEBUG | Message: Successfully indexed 4 documents!
            doc_idx@67[S]:artifacts of this executor (doc_idx_file) is persisted to /var/huforce/fs/workspace/doc_idx_file-0/doc_idx_file.bin
            doc_idx@67[I]:#sent: 3 #recv: 4 sent_size: 6.3 KB recv_size: 2.2 MB
           | ๐Ÿ“ƒ     16 โฑ๏ธ 262.9s ๐ŸŽ 0.1/s      4   requests        gateway@72[I]:send: 5 recv: 4 pending: 1
            crafter@60[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธโš
    DEBUG | Message: File-Info: research_admission.txt
    DEBUG | Message: File-Info: cloudcomputing.pptx
    DEBUG | Message: File-Info: WS_2012_White_Paper_Hyper-V.pdf
    DEBUG | Message: File-Info: writing-sql-queries.docx
            crafter@60[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            crafter@60[I]:#sent: 5 #recv: 6 sent_size: 1.2 MB recv_size: 5.2 KB
            encoder@63[I]:#sent: 4 #recv: 5 sent_size: 15.6 MB recv_size: 1.2 MB
          chunk_idx@66[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            encoder@63[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
          chunk_idx@66[I]:#sent: 4 #recv: 5 sent_size: 2.2 MB recv_size: 16.1 MB
    DEBUG | Message: Encoding 552 paragraphs
          chunk_idx@66[E]:ValueError('write to closed file')
     add "--quiet-error" to suppress the exception details
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 198, in _msg_callback
        self._zmqlet.send_message(self._callback(msg))
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 184, in _callback
        self._pre_hook(msg)._handle(msg)._post_hook(msg)
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 177, in _handle
        self._executor(self.request_type)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/__init__.py", line 595, in __call__
        d()
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/__init__.py", line 381, in __call__
        self._apply_all(documents, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/index.py", line 58, in _apply_all
        self.exec_fn(keys, values)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/decorators.py", line 44, in arg_wrapper
        f = func(self, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/indexers/keyvalue.py", line 178, in add
        self.write_handler.header.write(
    ValueError: write to closed file
            doc_idx@67[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโœ–โ–ธdoc_idx/ZEDRuntimeโ–ธโš
    dev_ai            | DEBUG | Message: Successfully indexed 4 documents!
    

    What's also strange is, that the failed documents for the chunk-idx are passed on to the doc-idx, as you can see in the logs.

    My flow for indexing:

    !AsyncFlow
    with:
      logserver: false
      show-exc-info: true
      prefetch: 2
      # request-size: 2
      # batch_size: 2
    pods:
      crafter:
        uses: pods/craft.yml
      encoder:
        uses: pods/encode.yml
        timeout_ready: 600000
      chunk_idx:
        uses: pods/chunk.yml
      doc_idx:
        uses: pods/doc.yml
    

    Originally posted by @nero-nazok in https://github.com/jina-ai/jina/issues/2298#issuecomment-817084516

    area/core type/bug 
    opened by JoanFM 32
  • DOCQA..

    DOCQA..

    I am trying to parse a pdf & apply the DOCQA model to it.Roberta as well as BERT..!! But I am not getting right answer for it.. whereas independently it's works accurately... !!

    This is my flow.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
    

    This is my query.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
      - name: generator
        uses: Generator
        py_modules: "flows/generator_roberta.py"
    
    opened by jyotikhetan 29
  • feat: allow multiple port and protocols for gateway

    feat: allow multiple port and protocols for gateway

    This PR adds support for multiple ports and protocols for the gateway runtime.

    • [x] Gateway parser can receive list or single value
    • [x] Flow __init__ and config_gateway can receive multiple or single value. Flow.port and Flow.protocol return single or multiple values
    • [x] GatewayRuntime expects list of values
    • [x] BaseGateway can receive runtime args
    • [ ] (optional) move injected dependencies to runtime_args, TBD in https://github.com/jina-ai/jina/issues/5396
    • [x] implemented Gateway classes uses the port and protocol from runtime args
    • [x] support in JAML
    • [x] k8s gateway service
    • [x] docker compose gateway service
    • [x] health checks
    • NetworkChecker
    • docker and k8s checks
    • Flow pod checks
    • [ ] ~~apply the same principle for port_monitoring~~ TBD in a separate PR: https://github.com/jina-ai/jina/issues/5352
    • [x] fix gateway output: https://github.com/jina-ai/jina/issues/5345
    • [ ] pass CI
    • [x] Add tests follow up: https://github.com/jina-ai/jina/issues/5393 closes: https://github.com/jina-ai/jina/issues/5345, https://github.com/jina-ai/jina/issues/5343
    size/M size/L area/core area/helper area/testing size/XL area/docs component/resource area/cli component/jaml 
    opened by alaeddine-13 28
  • Gateway error: received an empty stream from the client

    Gateway error: received an empty stream from the client

    Description

    I encountered an error in flow.post method while trying to run python app.py -t index in multires-lyrics-search after cloning this repository: examples. The error message was: gateway@61291[E]:receive an empty stream from the client! please check your client's inputs, you can use "Client.check_input(inputs)".


    Environment

    • jina 2.0.18
    • jina-proto 0.0.85
    • jina-vcs-tag (unset)
    • libzmq 4.3.4
    • pyzmq 1.21.5
    • protobuf 3.20.0
    • proto-backend cpp
    • grpcio 1.44.0
    • pyyaml 6.0
    • python 3.7.13
    • platform Linux
    • platform-release 5.10.105-1-MANJARO
    • platform-version #1 SMP PREEMPT Fri Mar 11 14:12:33 UTC 2022
    • architecture x86_64
    • processor
    • uid 5025763201072
    • session-id 7d472a12-b722-11ec-a806-049226d49030
    • uptime 2022-04-08T15:58:49.863145
    • ci-vendor (unset)
    • JINA_ARRAY_QUANT (unset)
    • JINA_CONTROL_PORT (unset)
    • JINA_DEFAULT_HOST (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_HUBBLE_REGISTRY (unset)
    • JINA_HUB_CACHE_DIR (unset)
    • JINA_HUB_ROOT (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_ID (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_LOG_WORKSPACE (unset)
    • JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
    • JINA_POD_NAME (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_VCS_VERSION (unset)
    • JINA_MP_START_METHOD (unset)
    opened by Jackal1586 28
  • feat: add grpc health checking

    feat: add grpc health checking

    This PR aims to change the way health check is done in the core for Executors but also for any runtime using grpc. We aim to use the standard grpc https://github.com/grpc/grpc/blob/master/doc/health-checking.md as suggested by grpc, and which is something that lots of Load Balancers, etc ... will use to make sure grpc-based microservices are healthy.

    This also has the benefit that ControlRequest can be removed from the code.

    area/core area/helper area/testing area/network size/XL area/docs component/resource area/setup component/client component/proto area/entrypoint component/type 
    opened by JoanFM 27
  • feat: remove head for non sharded deployments

    feat: remove head for non sharded deployments

    Remove heads for non sharded Deployments

    Description

    This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers. Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

    This is a breaking change :boom: It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

    Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

    • [x] Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD
    • [x] Set all worker Pods in the Gateway (Outside K8s)
    • [x] Make the gateway send to worker Pods
    • [x] Dont create heads for Deployments with no shards
    • [x] Implement reducing after multiple needs in the Gateway
    • [x] Adapt tests
    • [x] Change docs

    Closes #4503 (issue)

    area/core area/helper area/testing size/XL area/docs area/cli area/daemon 
    opened by jacobowitz 27
  • docs(contributing/callbacks): fix links and formatting

    docs(contributing/callbacks): fix links and formatting

    Some small fixes:

    • CONTRIBUTING - fix docs example links, remove "Alex" stuff

    • callbacks - add backticks around relevant text

    • README - fix links

    • [X] check and update documentation. See guide and ask the team.

    size/M size/S area/docs 
    opened by alexcg1 2
  • feat: expose deployment to serve Executors

    feat: expose deployment to serve Executors

    This PR makes it possible to serve Executors using Deployment

    • [x] Serve Executors with deployment
    • [x] add overload for Deployment
    • [ ] Unify behavior with Flow ?
      • print info
      • reload
      • refactor common code into BaseOrchestrator class
    • [ ] Tests: single Executor deployment (Executor python class, docker image, yaml), test for reload (if reload is implemented), dynamic batching (like in https://github.com/jina-ai/jina/blob/1fab6208a576b835f466fc7c7cac926b1b1cf185/tests/unit/serve/executors/test_executor.py#L49) ? in tests/unit/orchestrate/deployments/test_deployments.py or tests/integration/deployments/test_deployment.py
    • [x] Expose Deployment class
    • [ ] Documentation
    size/M size/L area/core area/helper area/testing area/network size/XL component/proto area/entrypoint 
    opened by alaeddine-13 5
  • Inconsistant embeddings from Executor

    Inconsistant embeddings from Executor

    Describe the bug JINA Executor from the hub is inconsistent using different methods. Having a dataset with 8000 examples of stable diffusion prompts, the embeddings generated from:

    # create Document Array
    da_custom = DocumentArray()
    for idx, row in df.iterrows():
        doc = Document(text=row['prompt'].strip(), tags={'topic': row['label_txt'],
                                                   'date': row['day'],
                                                'width': row['width'],
                                                'height': row['height'],
                                                        'image': row['image']})
        da_custom.append(doc)
    # Local Executor from the hub
    executor_custom.encode(da_custom, {})
    

    Differ from:

    # create Document Array
    da_custom = DocumentArray()
    # Loop through each item in the set, cada item del DataFrame es aรฑadido al DocumentArray
    for idx, row in df.iterrows():
        doc = Document(text=row['prompt'].strip(), tags={'topic': row['label_txt'],
                                                   'date': row['day'],
                                                'width': row['width'],
                                                'height': row['height'],
                                                        'image': row['image']})
        docs = DocumentArray(doc)
        executor_custom.encode(docs,{})
        doc.embedding = docs[0].embedding
        da_custom.append(doc)
    

    The embeddings would be exactly the same for every item in the docarray but, with the first method we get embeddings filled zeros in some examples:

    image

    While the second method that is applying exactly the same model on exactly the same text, we get:

    image

    As you can see they are cleary different but they use the same executor, this could greatly change results while developing, any ideas?

    -- UPDATE: I' ve also tried doing in batches of a 1000, it works, it seems the problem is encoding the whole thing ? Thought the batches logic is not that intuitive, any ideas? So far 2 methods have worked, the only difference is the volume being encoded with the executor.

    %%time
    
    da_custom = DocumentArray()
    
    factor = 1000
    df.groupby(np.arange(len(df))//factor)
    for k,g in df.groupby(np.arange(len(df))//factor):
        print(f'INDEX {k}')
        print(f'DF LEN {len(g)}')
        # Se crea arreglo de documentos (formato para bรบsqueda neuronal)
        da_tmp = DocumentArray(Document(text=x['prompt'].strip(), tags={'topic': x['label_txt'],'date': x['day'],
                                                                   'width': x['width'],'height': x['height'],
                                                                   'image': x['image']}) for idx, x in g.iterrows())
        executor_custom.encode(da_tmp,{})
        time.sleep(5)
        da_custom.extend(da_tmp)    
    

    Describe how you solve it I had to append DocumentArrays of length = 1 to encode them correctly.


    Environment

    • jina 3.13.0
    • docarray 0.20.1
    • jcloud 0.1.6
    • jina-hubble-sdk 0.28.0
    • jina-proto 0.1.13
    • protobuf 4.21.12
    • proto-backend upb
    • grpcio 1.47.2
    • pyyaml 5.4.1
    • python 3.10.9
    • platform Darwin
    • platform-release 21.6.0
    • platform-version Darwin Kernel Version 21.6.0: Wed Aug 10 14:28:23 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T6000
    • architecture arm64
    • processor arm
    • uid 73198290903005
    • session-id 64cd8050-876d-11ed-ba45-4292ce209fdd
    • uptime 2022-12-29T03:39:02.900723
    • ci-vendor (unset)
    • internal False
    • JINA_DEFAULT_HOST (unset)
    • JINA_DEFAULT_TIMEOUT_CTRL (unset)
    • JINA_DEPLOYMENT_NAME (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_EARLY_STOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_GATEWAY_IMAGE (unset)
    • JINA_GRPC_RECV_BYTES (unset)
    • JINA_GRPC_SEND_BYTES (unset)
    • JINA_HUB_NO_IMAGE_REBUILD (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_MP_START_METHOD (unset)
    • JINA_OPTOUT_TELEMETRY (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_LOCKS_ROOT (unset)
    • JINA_K8S_ACCESS_MODES (unset)
    • JINA_K8S_STORAGE_CLASS_NAME (unset)
    • JINA_K8S_STORAGE_CAPACITY (unset)
    • JINA_STREAMER_ARGS (unset)

    Screenshots Uploaded

    opened by bgonzalezfractal 1
  • Environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY not set by default on Mac

    Environment variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY not set by default on Mac

    The Jina installation script does not set the variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY to YES, while installing on my M1 Mac. As a result, after installation, when the user tries to run a flow locally, the Python interpreter process crashes while trying to start the server.

    How I solved it I solved it by exporting OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES on zsh.


    Environment

    • jina 3.13.1
    • docarray 0.20.1
    • jcloud 0.1.6
    • jina-hubble-sdk 0.29.0
    • jina-proto 0.1.13
    • protobuf 4.21.12
    • proto-backend upb
    • grpcio 1.51.1
    • pyyaml 6.0
    • python 3.11.1
    • platform Darwin
    • platform-release 22.1.0
    • platform-version Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:30 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T8103
    • architecture arm64
    • processor arm
    • uid 121687005276591
    • session-id 10c00bd8-820c-11ed-94a9-6eac76b929af
    • uptime 2022-12-22T16:19:44.863274
    • ci-vendor (unset)
    • internal False
    • JINA_DEFAULT_HOST (unset)
    • JINA_DEFAULT_TIMEOUT_CTRL (unset)
    • JINA_DEPLOYMENT_NAME (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_EARLY_STOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_GATEWAY_IMAGE (unset)
    • JINA_GRPC_RECV_BYTES (unset)
    • JINA_GRPC_SEND_BYTES (unset)
    • JINA_HUB_NO_IMAGE_REBUILD (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_MP_START_METHOD (unset)
    • JINA_OPTOUT_TELEMETRY (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_LOCKS_ROOT (unset)
    • JINA_K8S_ACCESS_MODES (unset)
    • JINA_K8S_STORAGE_CLASS_NAME (unset)
    • JINA_K8S_STORAGE_CAPACITY (unset)
    • JINA_STREAMER_ARGS (unset)

    Screenshots image

    opened by saahil 1
  • Create Deployment YAML separate from Executor YAML

    Create Deployment YAML separate from Executor YAML

    create deployment yaml separate from Executor yaml

    • add Deployment.load_config

    A deployment can use an Executor YAML like so: Deployment(uses='executor.yaml', replicas=...)

    But a deployment YAML is used like so: Deployment.load_config(uses='deployment.yaml')

    Possible usages of executors inside Flow YAML:

    1. executor config is embedded in config
     jtype: flow
     executors:
        jtype: Deployment
        uses:
            jtype: MyExecutor
            py_modules: executor.py
        replicas: 2
    
    1. old usage for backward compatibility:
     jtype: flow
     executors:
        - uses: executor.yaml
          replicas: 12
    
    1. simply deployment.yaml
     jtype: flow
     executors:
        - uses: deployment.yaml
          replicas: 2
          shard: 3
    

    Deployment args include protocol, shard, replicas, expose-gateway

    Executor args include the rest

    epic/executor 
    opened by alaeddine-13 0
Releases(v3.13.2)
  • v3.13.2(Dec 30, 2022)

    Release Note (3.13.2)

    This release contains 1 bug fix.

    ๐Ÿž Bug Fixes

    Respect timeout_ready when generating startup probe (#5560)

    As Kubernetes Startup Probes were added to all deployments in release v3.13.0, we added default values for all probe configurations. However, if those default configurations were not enough to wait for an Executor that takes time to load and become ready, the Executor deployment would become subject to the configured restart policy. Therefore, Executors that are slow to load would keep restarting forever.

    In this patch, this behavior is fixed by making sure that Startup Probe configurations respect the timeout_ready argument of Executors. Startup Probes are configured like so:

    • periodSeconds always set to 5 seconds
    • timeoutSeconds always set to 10 seconds
    • failureThreshold is the number of attempts maybe by kubernetes to check if the pod is ready. It varies according to timeout_ready. The formula used is failureThreshold = timeout_ready / 5000 (as timeout_ready is in microseconds and periodSeconds is 5 seconds) and in all cases, it will be at least 3. If timeout_ready is -1 (in Jina it means waiting forever for the Executor to become ready), since waiting forever is not supported in Kubernetes, the value for failureThreshold will be 120 attempts.

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.1(Dec 21, 2022)

    Release Note (3.13.1)

    Release time: 2022-12-21 12:58:15

    This release contains 3 bug fixes and 1 documentation improvement.

    ๐Ÿž Bug Fixes

    Support Gateway with multiple protocols for Kubernetes export (#5532)

    You can now export Flows with multiple protocols to Kubernetes. Previously this would cause an error.

    flow = Flow().config_gateway(protocol=['http', 'grpc'])
    flow.to_kubernetes_yaml('k8s_flow_folder')
    

    Fix Python 3.11 support (#5529)

    It was previously impossible to install Jina with Python 3.11 due to a grpcio dependency problem. grpcio added support for Python 3.11 only with version 1.49.0, causing potential problems when used by Jina and other projects.

    In this release grpcio>=1.49.0 is installed alongside Jina when using Python 3.11. However, be aware of potential problems related to grpc hanging.

    Unary RPC from Client respects results_in_order (#5513)

    In prior releases, calling the post method of a client with grpc and using stream=False did not respect the results_in_order parameter and results were always returned in order:

    # this wrongly returns results in order
    c = Client(protocol='grpc')
    c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)  
    

    Also this implied that using the Client with asyncio=True and stream=False in the post call would return results in the order that they were returned by the Flow, rather than respecting the input order:

    # this wrongly returns results in order
    c = Client(protocol='grpc', asyncio=True)
    
    async for resp in c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)
        print(resp)
    

    This release fixes the ordering bug.

    ๐Ÿ“— Documentation Improvements

    • Document inheritance of arguments from Flow API to Executors and Gateway (#5535)

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.0(Dec 15, 2022)

    Release Note (3.13.0)

    Release time: 2022-12-15 15:33:43

    This release contains 14 new features, 9 bug fixes and 7 documentation improvements.

    This release introduces major features like Custom Gateways, Dynamic Batching for Executors, development support with auto-reloading, support for the new namespaced Executor scheme jinaai, improvements for our gRPC transport layer, and more.

    ๐Ÿ†• Features

    Custom Gateways (#5153, #5189, #5342, #5457, #5465, #5472 and #5477)

    Jina Gateways are now customizable in the sense that you can implement them in much the same way as an Executor. With this feature, Jina gives power to the user to implement any server, protocol or interface at the Gateway level. There's no more need to build an extra service that uses the Flow.

    For instance, you can define a Jina Gateway that communicates with the rest of Flow Executors like so:

    from docarray import Document, DocumentArray
    from jina.serve.runtimes.gateway.http.fastapi import FastAPIBaseGateway
    
    
    class MyGateway(FastAPIBaseGateway):
        @property
        def app(self):
            from fastapi import FastAPI
    
            app = FastAPI(title='Custom FastAPI Gateway')
    
            @app.get(path='/service')
            async def my_service(input: str):
                # convert input request to Documents
                docs = DocumentArray([Document(text=input)])
    
                # send Documents to Executors using GatewayStreamer
                result = None
                async for response_docs in self.streamer.stream_docs(
                    docs=docs,
                    exec_endpoint='/',
                ):
                    # convert response docs to server response and return it
                    result = response_docs[0].text
    
                return {'result': result}
    
            return app
    

    Then you can use it in your Flow in the following way:

    flow = Flow().config_gateway(uses=MyGateway, port=12345, protocol='http')
    

    A Custom Gateway can be used as a Python class, YAML configuration or Docker image.

    Adding support for Custom Gateways required exposing the Gateway API and supporting multiple ports and protocols (mentioned in a prior release). You can customize it by subclassing the FastAPIBaseGateway class (for simple implementation) or base Gateway for more complex use cases.

    Working on this feature also involved exposing and improving the GatewayStreamer API as a way to communicate with Executors within the Gateway.

    Find more information in the Custom Gateway page.

    Dynamic batching (#5410)

    This release adds Dynamic batching capabilities to Executors.

    Dynamic batching allows requests to be accumulated and batched together before being sent to an Executor. The batch is created dynamically depending on the configuration for each endpoint.

    This feature is especially relevant for inference tasks where model inference is more optimized when batched to efficiently use GPU resources.

    You can configure Dynamic batching using either a decorator or the uses_dynamic_batching parameter. The following example shows how to enable Dynamic batching on an Executor that performs model inference:

    from jina import Executor, requests, dynamic_batching, Flow, DocumentArray, Document
    import numpy as np
    import torch
    
    
    class MyExecutor(Executor):
        def __init__(self, **kwargs):
            super().__init__(**kwargs)
    
            # initialize model
            self.model = torch.nn.Linear(in_features=128, out_features=128)
    
        @requests(on='/bar')
        @dynamic_batching(preferred_batch_size=10, timeout=200)
        def embed(self, docs: DocumentArray, **kwargs):
            docs.embeddings = self.model(torch.Tensor(docs.tensors))
    
    
    flow = Flow().add(uses=MyExecutor)
    

    With Dynamic Batching enabled, the Executor above will efficiently use GPU resources to perform inference by batching Documents together.

    Read more about the feature in the Dynamic Batching documentation page.

    Install requirements of local Executors (#5508)

    Prior to this release, the install_requirements parameter of Executors only installed Executor requirements for Hub Executors. Now, local Executors with a requirements.txt file will also have their requirements installed before starting Flows.

    Support jinaai Executor scheme to enable namespaced Hub Executors (#5462, #5468 and #5515)

    As Jina AI Cloud introduced namespaces to Executor resources, we made changes to support the new jinaai Executor scheme. This PR adds support for the new scheme.

    This means that namespaced Executors can now be used with the jinaai scheme in the following way:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai://jina-ai/DummyHubExecutor')
    

    This scheme is also supported in Kubernetes and other APIs:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai+docker://jina-ai/DummyHubExecutor')
    flow.to_kubernetes_yaml('output_path', k8s_namespace='my-namespace')
    

    The support of the new scheme means the minimum supported version of jina-hubble-sdk has been increased to 0.26.10.

    Add auto-reloading to Flow and Executor on file changes (#5461, #5488 and #5514)

    A new argument reload has been added to the Flow and Executor APIs, which automatically reloads running Flows and Executors when changes are made to Executor source code or YAML configurations of Flows and Executors.

    Although this feature is only meant for development, it aims to help developers iterate fast and automatically update Flows with changes they make live during development.

    Find out more about this feature in these two sections:

    Expand Executor serve parameters (#5494)

    The method Executor.serve can receive more parameters, similar to what the Flow API expects. With new parameters to control serving and deployment configurations of the Executor, this method empowers the Executor to be convenient for single service tasks.

    This means you can not only build advanced microservices-based pipelines and applications, but also build individual services with all Jina features: shards/replicas, dynamic batching, auto-reload, etc.

    Read more about the method in the Python API documentation.

    Add gRPC trailing metadata when logging gRPC error (#5512)

    When logging gRPC errors, context trailing metadata is now shown. This helps identify underlying network issues rather than the error codes that mask multiple network errors into a single gRPC status code.

    For instance, the new log message looks like the following:

    DEBUG  gateway@ 1 GRPC call to deployment executor0 failed                      
           with error <AioRpcError of RPC that terminated with:                     
                   status = StatusCode.UNAVAILABLE                                  
                   ...                                                                   
           trailing_metadata=Metadata((('content-length', '0'),                     
           ('l5d-proxy-error', 'HTTP Balancer service in                            
           fail-fast'), ('l5d-proxy-connection', 'close'),                          
           ('date', 'Tue, 13 Dec 2022 10:20:15 GMT'))), for                         
           retry attempt 2/3. Trying next replica, if available. 
    

    The trailing_metadata returned by load balancers will help to identify the root cause more accurately.

    Implement unary_unary stub for Gateway Runtime (#5507)

    This release adds the gRPC unary_unary stub for Gateway Runtime as a new communication stub with Executors. Since the gRPC performance best practices for Python page suggests that unary stream implementation might be faster for Python, we added this communication method.

    However, this is not enabled by default. The streaming RPC method will still be used unless you set the stream option to False in the Client.post() method. The feature is only effective when the gRPC protocol is used.

    Read more about the feature in the documentation: https://docs.jina.ai/concepts/client/send-receive-data/#use-unary-or-streaming-grpc

    Add Startup Probe and replace Readiness Probe with Liveness Probe (#5407)

    Before this release, when exporting Jina Flows to Kubernetes YAML configurations, Kubernetes Readiness Probes used to be added for the Gateway pod and each Executor pod. In this release we have added a Startup Probe and replaced Readiness Probe with Liveness Probe.

    Both probes use the jina ping command to check that pods are healthy.

    New Jina perf Docker images (#5498)

    We added a slightly larger Docker image with suffix perf which includes a set of tools useful for performance tuning and debugging.

    The new image is available in Jina AI's Docker hub.

    New Jina Docker image for Python 3.10, and use Python 3.8 for default Jina image (#5490)

    Besides adding Docker images aimed for performance optimization, we added an image with a newer Python version: 3.10. This is available in Jina AI's Docker hub, for example jinaai/jina:master-py310-standard.

    We also made Python 3.8 our minimum supported Python version by default, and it will be used for default Docker images.

    Minimize output of jina ping command (#5476)

    jina ping commands are now less verbose and will print less irrelevant output. However, important information like latency for each round, average latency, number of successful requests and ping result will still show up.

    Add Kubernetes preStop hook to the containers (#5445)

    A preStop hook has been added to Executors and the Gateway to allow a grace period. This allows more time to complete in-flight requests and finish the server's graceful shutdown.

    Generate random ports for multiple protocols (#5455)

    If you use multiple protocols for a Gateway, you no longer need to specify a port for each one. Whether it's Python or YAML, you just need to specify the protocols you want to support and Jina will generate random ports for you.

    Python API:

    from jina import Flow
    
    flow = Flow().config_gateway(protocol=['grpc', 'http', 'websocket'])
    with flow:
        flow.block()
    

    YAML:

    jtype: Flow
    gateway:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
    

    Result: flow

    ๐Ÿž Bug Fixes

    List-like args passed as string (#5464)

    We fixed the format expected for port, host and port_monitoring to feel more Pythonic. Basically, if you use replicas, you no longer have to provide comma-separated ports as a string value. Instead, you can simply pass a list of values, no need to put all in a string anymore!

    For instance, suppose we have two external replicas of an Executor that we want to join in our Flow (the first is hosted on localhost:12345 and the second on 91.198.174.192:12346). We can add them like this:

    from jina import Flow
    
    replica_hosts, replica_ports = ['localhost', '91.198.174.192'], [
        '12345',
        '12346',
    ]  # instead of 'localhost,91.198.174.192', '12345,12346'
    Flow().add(host=replica_hosts, port=replica_ports, external=True)
    

    Or:

    Flow().add(host=['localhost:12345', '91.198.174.192:12346'], external=True)
    

    Note that this is not a breaking change, and the old syntax (comma-separated values: Flow().add(host='localhost:12345,91.198.174.192:12346', external=True)) is still supported for backwards compatibility.

    Restore port to overload type hint and JSON schema (#5501)

    When we made port and protocol arguments of the Gateway support multiple values, a bug was introduced where port did not appear in Jina's JSON schema as well as the Flow API overload for method signatures.

    Although the arguments are functional in both the Python API and YAML, this suppressed auto-completion and developer support for these parameters. This release restores the port parameter in both the Flow method overloads and JSON schema.

    Do not force insecure to True in open telemetry integration (#5483)

    In Jina's instrumentation, communication to open telemetry exporters used to be forced to insecure mode. Luckily, our community member @big-thousand picked this up and submitted a fix. The communication is no longer forced to the insecure mode.

    Kudos to @big-thousand for his contribution!

    Fix problem when using floating Executor in HTTP (#5493)

    We found a bug when using Floating Executors in HTTP, where the floating Executor is connected to the Gateway (in the Flow topology). In this case, the Executor would not receive input Documents properly. This release fixes the mentioned bug.

    Add egg info post install command for egg info setup mode (#5491)

    This release adds support for the egg info setup mode in Python. This means post-installation commands are now properly executed in environments that rely on Python's new setup mode.

    This bug resulted in several issues especially for environments that depend on these post-installation commands. For instance, some Environment Variables that are needed for Jina to work on macOS and for CLI auto-complete.

    Do not apply limits when gpus='all' in Kubernetes (#5485)

    If Executor parameter gpus is set to "all", no limits will be applied on the pod in Kubernetes.

    Fix Windows signal handling (#5484)

    This release improves signal handling on Windows, specifically when cancelling a Flow with an OS signal.

    Cap opentelemetry-instrumentation-aiohttp-client (#5452)

    This release caps the version for opentelemetry-instrumentation-aiohttp-client which is incompatible with opentelemetry-semantic-conventions.

    Raise exceptions from path importer (#5447)

    Previously, errors were hidden when they came from a Python module imported to load an Executor. Actually the module was not considered to be a Python module, which produced other misleading errors. In this release, actual errors during imports will be raised and no longer hidden.

    ๐Ÿ“— Documentation Improvements

    • Add gRPC requirements for Apple Silicon (M1 Chip) to fix failing installation of Jina (#5511)
    • Add redirects from '/fundamentals' to '/concepts' (#5504)
    • Update JCloud documentation to the jcloud v0.1.0 (#5385)
    • Restructure documentation under /concepts
    • Change Executor URI scheme to namespaced scheme jinaai (#5450)
    • Custom Gateway documentation (#5465)
    • Provide more accurate description for port and protocol parameters of the Gateway (#5456)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Delgermurun (@delgermurun)
    • Jie Fu (@jemmyshin)
    • Alex Cureton-Griffiths (@alexcg1)
    • big-thousand (@big-thousand)
    • IyadhKhalfallah (@IyadhKhalfallah)
    • Deepankar Mahapatro (@deepankarm)
    • samsja (@samsja)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Joan Fontanals (@JoanFM)
    • Anne Yang (@AnneYang720)
    • Han Xiao (@hanxiao)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    Source code(tar.gz)
    Source code(zip)
  • v3.12.0(Nov 25, 2022)

    Release Note (3.12.0)

    Release time: 2022-11-25 12:28:29

    This release contains 8 new features, 16 bug fixes and 15 documentation improvements.

    ๐Ÿ†• Features

    Support multiple protocols at the same time in Flow Gateways (#5435 and #5378)

    Prior to this release, a Flow only exposed one server in its Gateway with one of the following protocols: HTTP, gRPC or WebSockets.

    Now, you can specify multiple protocols and for each one, a separate server is started. Each server is bound to its own port.

    For instance, you can do:

    from jina import Flow
    flow = Flow(port=[12345, 12346, 12347], protocol=['http', 'grpc', 'websocket'])
    with flow:
        flow.block()
    

    or: jina flow --uses flow.yml where flow.yml is:

    jtype: Flow
    with:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
      port:
        - 12345
        - 12344
        - 12343
    

    multi-protocol-flow

    The protocol and port parameters can still accept single values rather than a list. Therefore, there is no breaking change.

    Alias parameters protocols and ports are also defined:

    flow = Flow(ports=[12345, 12346, 12347], protocols=['http', 'grpc', 'websocket'])
    

    In Kubernetes, this exposes separate services for each protocol.

    Read the docs for more information.

    Add Kubernetes information to resource attributes in instrumentation (#5372)

    When deploying to Kubernetes, the Gateway and Executors expose the following Kubernetes information as resource attributes in instrumentation:

    • k8s.namespace.name
    • k8s.pod.name
    • k8s.deployment.name / k8s.statefulset.name

    Besides that, the following resource attributes are set if they are present in the environment variables of the container:

    • k8s.cluster.name (set K8S_CLUSTER_NAME environment variable)
    • k8s.node.name (set K8S_NODE_NAME environment variable)

    Add option to return requests in order using the Client (#5404)

    If you use replicated Executors, those which finish processing first return their results to the Gateway which then returns them to the client. This is useful if you want results as soon as each replicated Executor finishes processing your Documents.

    However, this may be inconvenient if you want the Documents you send to the Flow to return in order. In this release, you can retain the order of sent Documents (when using replicated Executors) by passing the results_in_order parameter in the Client.

    For instance, if your Flow looks like this:

    from jina import Flow, DocumentArray, Document
    f = Flow().add(replicas=2)
    

    You can do the following to keep results in order:

    input_da = DocumentArray([Document(text=f'{i}') for i in range(100)])
    
    with f:
        result_da = f.post('/', inputs=input_da, request_size=10,  results_in_order=True)
        assert result_da[:, 'text'] == input_da[:, 'text']
    

    Add docs_map parameter to Executor endpoints (#5366)

    Executor endpoint signatures are extended to the following:

    class MyExecutor(Executor):
        @requests
        async def foo(
            self, docs: DocumentArray, parameters: Dict, docs_matrix: Optional[List[DocumentArray]], docs_map: Optional[Dict[str, DocumentArray]]
        ) -> Union[DocumentArray, Dict, None]:
            pass
    

    Basically, the parameter docs_map has been added. It's a dictionary that maps previous Executor names to DocumentArrays. This is useful when you have an Executor that combines results from many previous Executors, and you need information about where each resulting DocumentArray comes from.

    Add Gateway API (#5342)

    Prior to this release, all Gateway configurations were specified in the Flow API. However, by principle, Flow parameters are commonly inherited by Executors and the Gateway. We already gave the Executor its own API to be customized (either using the method add() or the executors YAML section in Flow YAML).

    In this release, we have done the same for Gateway. It defines its own API in both the Python API and YAML interface. In the Python API, you can configure the Gateway using the config_gateway() method:

    flow = Flow().config_gateway(port=12345, protocol='http')
    

    And in the YAML interface, you can configure the Gateway using the gateway section:

    !Flow
    gateway:
      protocol: http
      port: 12344
    executors:
      - name: exec
    

    This is useful when you want to apply parameters just for the Gateway. If you want a parameter to be applied to all Executors, then continue to use the Flow API.

    Keep in mind that you can still provide Gateway parameters using the Flow API. This means there are no breaking changes introduced.

    Support UUID in CUDA_VISIBLE_DEVICES round-robin assignment (#5360)

    You can specify a comma-separated list of GPU UUIDs in the CUDA_VISIBLE_DEVICES to assign devices to Executor replicas in a round-robin fashion. For instance:

    CUDA_VISIBLE_DEVICES=RRGPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5,GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5,GPU-0ccccccc-74d2-7297-d557-12771b6a79d5,GPU-0ddddddd-74d2-7297-d557-12771b6a79d5
    

    Check CUDA's documentation to see the accepted formats to assign CUDA devices by UUID.

    | GPU device | Replica ID | |------------------------------------------|------------| | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 0 | | GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5 | 1 | | GPU-0ccccccc-74d2-7297-d557-12771b6a79d5 | 2 | | GPU-0ddddddd-74d2-7297-d557-12771b6a79d5 | 3 | | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 4 |

    Thanks to our community member @mchaker for submitting this feature request!

    Capture shard failures in the head runtime (#5338)

    In case you use Executor shards, partially failed requests (those that fail on a subset of the shards) no longer raise an error.

    Instead, successful results are returned. An error is raised only when all shards fail to process Documents. Basically, the HeadRuntime's behavior is updated to fail only when all shards fail.

    Thanks to our community user @soumil1 for submitting this feature request.

    Add successful, pending and failed metrics to HeadRuntime (#5374)

    More metrics have been added to the Head Pods:

    • jina_number_of_pending_requests: number of pending requests
    • jina_successful_requests: number of successful requests
    • jina_failed_requests: number of failed requests
    • jina_received_request_bytes: the size of received requests in bytes
    • jina_sent_response_bytes: the size of sent responses in bytes

    See more in the instrumentation docs.

    Add deployment label in grpc stub metrics (#5344)

    Executor metrics used to show up aggregated at the Gateway level and users couldn't see separate metrics per Executor. With this release, we have added labels for Executors so that metrics in the Gateway can be generated per Executor or aggregated over all Executors.

    ๐Ÿž Bug Fixes

    Check whether the deployment is in Executor endpoints mapping (#5440)

    This release adds an extra check in the Gateway when sending requests to deployments: The Gateway sends requests to the deployment only if it is in the Executor endpoint mapping.

    Unblock event loop to allow health service (#5433)

    Prior to this release, sync function calls inside Executor endpoints blocked the event loop. This meant that health-checks submitted to Executors failed for long tasks (for instance, inference using a large model).

    In this release, such tasks no longer block the event loop. While concurrent requests to the same Executor wait until the sync task finishes, other runtime tasks remain functional, mainly health-checks.

    Dump environment variables to string for Kubernetes (#5430)

    Environment variables are now cast to strings before dumping them to Kubernetes YAML.

    Unpin jina-hubble-sdk version (#5412)

    This release frees (unpins) jina-hubble-sdk version. The latest jina-hubble-sdk is installed with the latest Jina.

    Bind servers to host argument instead of __default_host__ (#5405)

    This release makes servers at each Jina pod (head, Gateway, worker) bind to the host address specified by the user, instead of always binding to the __default_host__ corresponding to the OS. This lets you, depending on your network interface, restrict or expose your Flow services in your network.

    For instance, if you wish to expose all pods to the internet, except for the last Executor, you can do:

    flow = Flow(host='0.0.0.0').add().add(host='127.0.0.1')
    

    After this fix, Jina respects this syntax and binds the last Executor only to 127.0.0.1 (accessible only inside the host machine).

    Thanks to @wqh17101 for reporting this issue!

    Fix backoff_multiplier format when using max_attempts in the Client (#5403)

    This release fixes the format of backoff_multiplier parameter when injected into the gRPC request. The issue appeared when you use the max_attempts parameter in Client.

    Maintain the correct tracing operations chain (#5391)

    Tracing spans for Executors used to show up out of order. This behavior has been fixed by using the method start_as_current_span instead of start_span to maintain the tracing chain in the correct order.

    Use Async health servicer for tracing interceptors when tracing is enabled (#5392)

    When tracing is enabled, health checks in Docker and Kubernetes deployments used to fail silently until the Flow timed out. This happened because tracing interceptors expected RPC stubs to be coroutines.

    This release fixes this issue by using the async aio.HealthServicer instead of grpc_health.HealthServicer. Health checks submitted to runtimes (Gateway, head, worker) no longer fail when tracing is enabled.

    Properly update requests count in case of Exception inside the HeadRuntime (#5383)

    In case of an Exception being raised in the HeadRuntime, request counts were not updated properly (pending requests should have been decremented and failed requests should have been incremented). This is fixed in this release and the Exception is caught to update request counts.

    Fix endpoint binding when inheriting Executors (#5380)

    When an Executor is inherited, the bound endpoints of the parent Executor used to be overridden by those of the child Executor. This meant, if you inherited from Executors but still chose to use the parent Executor in your Flow, a wrong endpoint could have been called. This behavior is fixed by making Executor.requests a nested dict that also includes information about the class name. This helps to properly support Executor inheritance.

    Missing recording logic in connection stub metrics (#5363)

    Recording of request and response size in bytes is fixed to track all cases. This makes these metrics more accurate for the Gateway.

    Move build configs to pyproject (#5351)

    Build requirements have been moved from setup.py to pyproject.toml. This suppresses deprecation warnings that show up when installing Jina.

    New timer should keep labels (#5341)

    The MetricsTimer in instrumentation previously created new timers without keeping the histogram metric labels. This behavior is fixed and new timers retain the same labels.

    Use non-mutable default for MetricsTimer constructor (#5339)

    Use None instead of empty dict as a default value for histogram_metric_labels in MetricsTimer constructor.

    Catch RpcErrors and show better error messages in Client (#5325)

    In the Client, we catch RpcError and show its details instead of showing a standard error message.

    Import OpenTelemetry functions only when tracing is enabled in WorkerRuntime (#5321)

    This release ensures OpenTelemetry functions are only imported when tracing is enabled in the worker.

    ๐Ÿ“— Documentation Improvements

    • Remove 3 off-topic articles
    • Enable flag to convert resource labels to metric labels (#5409)
    • Add reference to Jina Kotlin client from community (#5390)
    • Add section about the transition from DocArray to Jina (#5382)
    • Add tips about supporting multiprocessing with fork in Jina when using macOS (#5379)
    • Fix reference to multiprocessing with spawn section and emphasize the need for entrypoint protection (#5370)
    • Update instructions to build protos locally using protogen Docker image (#5335)
    • Change mentions of JCloud to Jina AI Cloud (#5329)
    • Restructure docs into cloud-native section (#5332)
    • Add contributor acknowledgement and spell checking (#5324)
    • Create a Kubernetes section (#5315)
    • Add reference to Go and PHP clients (#5253)
    • Introduce versioning to the documentation (#5310)
    • Support redirects for removed documentation pages (#5301)
    • Use better Grafana screenshot without random text block (#5306)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Ziniu Yu (@ZiniuYu)
    • Andrei Ungureanu (@Andrei997)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Joan Fontanals (@JoanFM)
    • Zhaofeng Miao (@mapleeit)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    • Han Xiao (@hanxiao)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Nan Wang (@nan-wang)
    Source code(tar.gz)
    Source code(zip)
  • v3.11.2(Nov 22, 2022)

    Release note

    This release contains 1 hot fix.

    ๐Ÿž Bug Fix

    Unpin jina-hubble-sdk

    To avoid conflicting dependency versions with docarray, unpin the jina-hubble-sdk version dependency

    ๐ŸคŸ Contributors

    Joan Fontanals (@JoanFM )

    Source code(tar.gz)
    Source code(zip)
  • v3.11.0(Oct 24, 2022)

    Release Note (3.11.0)

    Release time: 2022-10-24 14:42:35

    This release contains 6 new features, 1 bug fix and 10 documentation improvements.

    ๐Ÿ†• Features

    Add OpenTelemetry tracing and metrics with basic configuration (#5175)

    Jina now supports OpenTelemetry Tracing and Metrics libraries for increased observability and instrumentation of Jina Runtimes. Read the docs (#5291) for details. We also provide a migration guide for a smooth transition from the previously-supported Prometheus-only based metrics instrumentation.

    You can integrate Jaeger or other distributed tracing tools to collect and visualize request-level and application level service operation attributes. This helps you analyze request-response lifecycle, application behavior and performance.

    from jina import Executor, requests, DocumentArray
    
    class MyExec(Executor):
        @requests
        def encode(self, docs: DocumentArray, **kwargs):
            with self.tracer.start_as_current_span(
                'encode', context=tracing_context
            ) as span:
                with self.monitor(
                    'preprocessing_seconds', 'Time preprocessing the requests'
                ):
                    docs.tensors = preprocessing(docs)
                with self.monitor(
                    'model_inference_seconds', 'Time doing inference the requests'
                ):
                    docs.embedding = model_inference(docs.tensors)
    

    Record existing Prometheus metrics into OpenTelemetry histograms (#5275)

    Prometheus-only based metrics are also available as OpenTelemetry supported metrics. You can switch to OpenTelemetry metrics with little effort.

    Add default tracing interceptors to head gRPC connection pool (#5271)

    Head Runtime supports default traces when forwarding requests to the shard replica.

    Add gRPC metadata to Executors and Gateway deployments (#5221)

    You can now provide HTTP header access tokens as client metadata for accessing external and authentication protected services.

    from jina import Client
    
    client = Client(host='api.clip.jina.ai', port=2096, tls=True, protocol='grpc')
    client.post(on='/encode', metadata=(('authorization', '<your access token>'),))
    

    Show local location of Executors in Hub (#5282)

    Use the list sub command to list the locations of local Hub Executors:

    jina hub list
    

    asciicast

    Dump to Statefulset in K8s when volumes are passed to Executor (#5265)

    Jina will generate a Kubernetes Statefulset manifest for Executors if you provide volumes to the Flow.to_kubernets_yaml() method. Executors can persist data on mounted volumes and correctly scale horizontally if required.

    ๐Ÿž Bug Fixes

    Invalid input raises exception (#5141)

    • Theclient.post() method was not raising an exception if any of the underlying async requests raised an exception.
    • The new version correctly raises the exception raised by any async request. The user can now correctly catch any exceptions and act upon the exception.

    ๐Ÿ“— Documentation Improvements

    • Docs for OpenTelemetry instrumentation (#5291)
    • Add Jina AI Cloud restructure Jcloud and Hub (#5298)
    • Standardize naming conventions (#5285)
    • Fix metrics naming (#5277)
    • Misc clean up (#5255, #5287, #5284, #5290, #5289, #5288)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Anthony Le (@AnthonyLe93)
    • felix-wang (@numb3r3)
    • Girish Chandrashekar (@girishc13)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Han Xiao (@hanxiao)
    • Alex Cureton-Griffiths (@alexcg1)
    • Andrei Ungureanu (@Andrei997)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.1(Oct 6, 2022)

    Release Note (3.10.1)

    Release time: 2022-10-06 15:05:48

    This release contains 1 new feature, 1 refactor, 3 bug fixes and 7 documentation improvements.

    ๐Ÿ†• Features

    Distributed replicas across different hosts (#5217)

    You can now start different Executors on different machines and use them as replicas by passing their addresses in the host parameter as a list, together with the external flag set to True:

    f.add(host='localhost:12345,91.198.174.192:12346', external=True)
    

    The Flow ensures the requests are load balanced between Executor instances.

    โš™ Refactoring

    Move Hubble related code to hubble-sdk (#5227)

    All code related to hubble and communication with Jina Hub has been moved to the jina-hubble-sdk repo and has been added as an external dependency.

    ๐Ÿž Bug Fixes

    Disable rich traceback in logging (#5242)

    We disabled the use of rich logging when printing tracebacks for Exceptions. This makes it easier to debug issues as the lines are not broken.

    Previously, an Exception would be reported to the console as:

      ERROR  executor0/rep-0@183551 ZeroDivisionError('division by [10/05/22 11:26:23]
           zero')                                                                   
            add "--quiet-error" to suppress the exception                           
           details                                                                  
           โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Traceback (most recent call last) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in process_data                                   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   179 โ”‚   โ”‚   โ”‚   โ”‚   if self.logger.debug_enable โ”‚                    
           โ”‚   180 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._log_data_request( โ”‚                    
           โ”‚   181 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚ โฑ 182 โ”‚   โ”‚   โ”‚   โ”‚   result = await self._data_r โ”‚                    
           โ”‚   183 โ”‚   โ”‚   โ”‚   โ”‚   if self._successful_request โ”‚                    
           โ”‚   184 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._successful_reques โ”‚                    
           โ”‚   185 โ”‚   โ”‚   โ”‚   โ”‚   return result               โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in handle                                         โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   161 โ”‚   โ”‚   )                                   โ”‚                    
           โ”‚   162 โ”‚   โ”‚                                       โ”‚                    
           โ”‚   163 โ”‚   โ”‚   # executor logic                    โ”‚                    
           โ”‚ โฑ 164 โ”‚   โ”‚   return_data = await self._executor. โ”‚                    
           โ”‚   165 โ”‚   โ”‚   โ”‚   req_endpoint=requests[0].header โ”‚                    
           โ”‚   166 โ”‚   โ”‚   โ”‚   docs=docs,                      โ”‚                    
           โ”‚   167 โ”‚   โ”‚   โ”‚   parameters=params,              โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall__                                      โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   290 โ”‚   โ”‚   if req_endpoint in self.requests:   โ”‚                    
           โ”‚   291 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   292 โ”‚   โ”‚   elif __default_endpoint__ in self.r โ”‚                    
           โ”‚ โฑ 293 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   294 โ”‚                                           โ”‚                    
           โ”‚   295 โ”‚   async def __acall_endpoint__(self, req_ โ”‚                    
           โ”‚   296 โ”‚   โ”‚   func = self.requests[req_endpoint]  โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall_endpoint__                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   311 โ”‚   โ”‚   โ”‚   if iscoroutinefunction(func):   โ”‚                    
           โ”‚   312 โ”‚   โ”‚   โ”‚   โ”‚   return await func(self, **k โ”‚                    
           โ”‚   313 โ”‚   โ”‚   โ”‚   else:                           โ”‚                    
           โ”‚ โฑ 314 โ”‚   โ”‚   โ”‚   โ”‚   return func(self, **kwargs) โ”‚                    
           โ”‚   315 โ”‚                                           โ”‚                    
           โ”‚   316 โ”‚   @property                               โ”‚                    
           โ”‚   317 โ”‚   def workspace(self) -> Optional[str]:   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in arg_wrapper                                    โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   159 โ”‚   โ”‚   โ”‚   โ”‚   def arg_wrapper(            โ”‚                    
           โ”‚   160 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   executor_instance, *arg โ”‚                    
           โ”‚   161 โ”‚   โ”‚   โ”‚   โ”‚   ):  # we need to get the su โ”‚                    
           โ”‚       the self                                    โ”‚                    
           โ”‚ โฑ 162 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   return fn(executor_inst โ”‚                    
           โ”‚   163 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚   164 โ”‚   โ”‚   โ”‚   โ”‚   self.fn = arg_wrapper       โ”‚                    
           โ”‚   165                                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/USER/.config/JetBrains/PyCharmCE2022.2/scrโ€ฆ โ”‚                    
           โ”‚ in foo                                            โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   12 โ”‚                                            โ”‚                    
           โ”‚   13 โ”‚   @requests                                โ”‚                    
           โ”‚   14 โ”‚   def foo(self, docs, **kwargs):           โ”‚                    
           โ”‚ โฑ 15 โ”‚   โ”‚   1/0                                  โ”‚                    
           โ”‚   16                                              โ”‚                    
           โ”‚   17 with Flow().add(uses=MyExec) as f:           โ”‚                    
           โ”‚   18                                              โ”‚                    
           โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ                    
           ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
      File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
        f.search(inputs=[Document()])
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
        return run_async(
      File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
        return asyncio.run(func(*args, **kwargs))
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
        async for resp in c._get_results(*args, **kwargs):
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
        callback_exec(
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
        raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "c50a57014fb948ccbd8065ada487a136"
    status {
      code: ERROR
      description: "ZeroDivisionError(\'division by zero\')"
      exception {
        name: "ZeroDivisionError"
        args: "division by zero"
        stacks: "Traceback (most recent call last):\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
        stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
        stacks: "ZeroDivisionError: division by zero\n"
        executor: "MyExec"
      }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    After the fix, the error is much clearer:

    ERROR  executor0/rep-0@198650 ZeroDivisionError('division by [10/05/22 11:40:37]
         zero')                                                                   
          add "--quiet-error" to suppress the exception                           
         details                                                                  
         Traceback (most recent call last):                                       
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 182, in process_data                                                
             result = await                                                       
         self._data_request_handler.handle(requests=requests)                     
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 164, in handle                                                      
             return_data = await self._executor.__acall__(                        
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 293, in __acall__                                                   
             return await                                                         
         self.__acall_endpoint__(__default_endpoint__,                            
         **kwargs)                                                                
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 314, in __acall_endpoint__                                          
             return func(self, **kwargs)                                          
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 162, in arg_wrapper                                                 
             return fn(executor_instance, *args, **kwargs)                        
           File                                                                   
         "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratcโ€ฆ                    
         line 15, in foo                                                          
             1/0                                                                  
         ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
    File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
      f.search(inputs=[Document()])
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
      return run_async(
    File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
      return asyncio.run(func(*args, **kwargs))
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
      return loop.run_until_complete(main)
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
      return future.result()
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
      async for resp in c._get_results(*args, **kwargs):
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
      callback_exec(
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
      raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "5d5e0338d0cd4bdd89efc430bbaaa74d"
    status {
    code: ERROR
    description: "ZeroDivisionError(\'division by zero\')"
    exception {
      name: "ZeroDivisionError"
      args: "division by zero"
      stacks: "Traceback (most recent call last):\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
      stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
      stacks: "ZeroDivisionError: division by zero\n"
      executor: "MyExec"
    }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    Remove lambda function in logger (#5249)

    We removed a reference to a lambda function in the JinaLogger so that there is no problem pickling it:

    import pickle
    
    from jina import Executor
    
    class MyExecutor(Executor):
    
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
    
    executor = MyExecutor()
    
    with open('executor.pickle', 'wb') as f:
        pickle.dump(executor, f)
    

    Previously, an Executor could not be pickled because it had a reference to a JinaLogger.

    This used to raise an AttributeError exception:

    Traceback (most recent call last):
      File "/home/USER/toy.py", line 14, in <module>
        pickle.dump(executor, f)
    AttributeError: Can't pickle local object 'JinaLogger.__init__.<locals>.<lambda>'
    

    Fix gRPC fork support (#5250)

    We enabled proper fork support for gRPC to correctly clean up gRPC usage in parent and child processes. This prevents potential issues when forking Executor processes.

    ๐Ÿ“— Documentation Improvements

    • Explain Executor patching in Kubernetes (#5235)
    • Add note about keeping same Jina versions across microservices (#5239)
    • Add missing request numbers in sample code (#5237)
    • JCloud YAML reorder (#5234)
    • Docs spelling/grammar/punctuation (#5244) (#5240)
    • Fix typos (#5236)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Sami Jaghouar (@samsja)
    • Alex Cureton-Griffiths (@alexcg1)
    • Johannes Messner (@JohannesMessner)
    • Delgermurun (@delgermurun)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Jackmin801 (@Jackmin801)
    • Joan Fontanals (@JoanFM)
    • Girish Chandrashekar (@girishc13)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.0(Sep 29, 2022)

    Release Note (v3.10.0)

    Release time: 2022-09-29 16:18:19

    This release contains 10 new features, 9 bug fixes and 12 documentation improvements.

    ๐Ÿ†• Features

    Enable jina ping as readiness probe on local and Kubernetes (#5200)

    This feature lets you check the readiness of an entire Flow or its individual components:

    jina ping flow grpc://localhost:12345
    
    jina ping executor localhost:12346
    
    jina ping gateway grpc://localhost:12345
    

    Successful output:

    INFO   JINA@28600 readiness check succeeded 1 times!!! 
    

    Unsuccessful output:

    This command exits with code 1 when the readiness check is not successful. Therefore, it is a good choice to use as a readiness probe for Executor and Gateway when deployed in Kubernetes.

    Add argument to specify exceptions that will exit the runtime (#5165)

    You can specify which Exceptions should trigger termination of an affected Executor:

    Flow().add(uses=MyExecutor, exit_on_exceptions: ['Exception',  'RuntimeException'])
    

    This ensures an Executor is not stuck in an unusable but technically alive state, and that the Executor can be restarted. This is especially useful when Executor restarts are automated through Kubernetes.

    Retry mechanism for client.post (#5176)

    client.post() accepts max_attempts, initial_backoff, max_backoff and backoff_multiplier parameters to control the capacity to retry requests, when a transient connectivity error occurs, using an exponential backoff strategy:

    client.post(
        on='/',
        inputs=docs,
        max_attempts=5,
        initial_backoff=0.1,
        max_backoff=0.5,
        backoff_multiplier=1.5
    )
    

    Add monitoring of requests size everywhere (#5111)

    The size of requests (jina_request_size_bytes) is now monitored everywhere: On every Executor, every Gateway, and every sharding Head.

    Consequently, this new metric can now be accessed via Prometheus and Grafana.

    JSON logging for JCloud (#5201)

    It is now possible to create logs in JSON format, instead of the standard line-by-line text output.

    Sample output:

    {"created": 1663930387.5850368, "filename": "data_request_handler.py", "funcName": "_load_executor", "levelname": "DEBUG", "lineno": 98, "module": "data_request_handler", "msg": "<jina.serve.executors.BaseExecutor object at 0x7fc1951f78d0> is successfully loaded!", "name": "WorkerRuntime", "pathname": "/home/foo/jina/serve/runtimes/request_handlers/data_request_handler.py", "process": 13386, "processName": "Pod", "thread": 140469860276032, "threadName": "MainThread"}
    

    To enable this, set the environment variable 'JINA_LOG_CONFIG to 'json'.

    Support list-like syntax to specify visible cuda devices (#5187)

    You can now use device id's to specify visible CUDA devices that should be used in the round-robin GPU assignment:

    `CUDA_VISIBLE_DEVICES=RR1,3`
    

    This will assign CUDA devices 1 and 3 to the different Executor replicas in a round-robin fashion.

    Make jina hub push non-blocking (#5129)

    jina hub push is now a non-blocking operation, meaning that the user does not have to wait for the entire push to be completed on the server side.

    While the push is being processed in the background, its status can be checked with the following command:

    jina hub status [<path_to_executor_folder>] [--id TASK_ID] [--verbose] [--replay]
    

    Warning messages in jina hub push (#5156)

    jina hub push can now display helpful warning messages, such as deprecation warnings: Screen Shot 2022-09-09 at 16 32 58

    Add duration info in events (#5157)

    Telemetry data now also includes the duration of every event that is reported.

    ๐Ÿž Bug Fixes

    Fix compatibility with Protobuf Python backend (#5222)

    This bug caused an exception when Python was used as a backend for Protobuf, rather than cpp or udp.

    With this fix, Python can also be used as a backend for Protobuf.

    Move significant params to sequential (#5205)

    This bug (occasionally) caused docker.errors.NullResource: Resource ID was not provided when pulling from Jina Hub. This was due to a faulty local caching implementation, which is now fixed.

    Fix missing secret when logged-in user with --force-update (#5180)

    This bug caused a failure for logged in users when force pushing to Jina Hub: Their secret would not be passed. This is now fixed and force push works as expected.

    Use default gRPC parameters for gRPC connection pool connection (#5211)

    We brought our gRPC client in line with the recommended configuration at https://github.com/grpc/grpc/blob/master/doc/keepalive.md

    Bump DocArray version for new column syntax (#5171)

    Some changes to the column syntax in DocArray required a version bump.

    Increase minimum Protobuf version (#5166)

    Some changes in the Protobuf library required a version bump.

    Remove leftover prints (#5199)

    Some unwanted print statements had sneaked their way into our codebase, and we kicked them right back out!

    ๐Ÿ“— Documentation Improvements

    • Change setenv into environ (#5223)
    • Mention prefetch (#5220)
    • Fix install instructions docker (#5213)
    • Fix grammar, wording, punctuation (#5209)
    • Fix grammar, punctuation (#5208)
    • Clarify exec endpoint usage in http (#5202)
    • Document grpc client limitation (#5193)
    • Update faq and lifetime (#5191)
    • Improve warning about config file in custom docker (#5190)
    • Clarify return behavior in parameters (#5179)
    • Add section for exit_on_exceptions argument (#5172)
    • Labels in flow yaml (#5164)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Girish Chandrashekar (@girishc13)
    • Andrei Ungureanu (@Andrei997)
    • Michael Gรผnther (@guenthermi)
    • Deepankar Mahapatro (@deepankarm)
    • Han Xiao (@hanxiao)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Nikolas Pitsillos (@npitsillos)
    • samsja (@samsja)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.3(Sep 29, 2022)

    Release Note (3.9.3)

    Release time: 2022-09-29 16:05:04

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, AlaeddineAbdessalem, Michael Gรผnther, Yanlong Wang, samsja, Alex Cureton-Griffiths, Deepankar Mahapatro, Nikolas Pitsillos, Han Xiao, zhangkai, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [11533962] - add gateway option to jina ping and use ping in readinessProbe (#5200) (Joan Fontanals)
    • [d0838d37] - add multiple attempts options to client.post API (#5176) (Joan Fontanals)
    • [13edf908] - use default grpc parameters for grpc connection pool connection (#5211) (samsja)
    • [216b4bf0] - monitoring: add monitoring of requests size in bytes at all level (#5111) (samsja)
    • [737875f5] - logs: json logging for jcloud (#5201) (Deepankar Mahapatro)
    • [fd4c0347] - support list-like syntax to round robin CUDA devices (#5187) (Joan Fontanals)
    • [9dff4c88] - add duration info in events (#5157) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [3683fccf] - fix compatibility with protobuf python backend (#5222) (AlaeddineAbdessalem)
    • [e932d6f9] - hubio.fetch_meta: move significant params to sequential (#5205) (Yanlong Wang)
    • [60e9b8de] - remove leftover prints (#5199) (Joan Fontanals)
    • [da186a23] - provide logger and streamer (#5188) (AlaeddineAbdessalem)
    • [7b5c0316] - fix get-openapi-schemas script (#5185) (AlaeddineAbdessalem)
    • [caf4a3d6] - fix missing secret when logged-in user with --force-update and โ€ฆ (#5180) (zhangkai)

    ๐Ÿงผ Code Refactoring

    • [9a6079ef] - separate gateway and asyncio runtime readiness checks (#5224) (AlaeddineAbdessalem)
    • [243639dd] - extract gateway app logic into custom gateway class (#5153) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [147317db] - change setenv into environ (#5223) (Michael Gรผnther)
    • [ad37bdd2] - mention prefetch (#5220) (Joan Fontanals)
    • [b9c8e44e] - fix install instructions docker (#5213) (Joan Fontanals)
    • [7cbf3471] - what-is-jina: fix grammar, wording, punctuation (#5209) (Alex Cureton-Griffiths)
    • [7c7d2cb0] - what-is-modality: fix grammar, punctuation (#5208) (Alex Cureton-Griffiths)
    • [1399e36c] - clarify exec endpoint usage in http (#5202) (Joan Fontanals)
    • [842a585b] - document grpc client limitation (#5193) (AlaeddineAbdessalem)
    • [96c5e7c1] - jcloud: update faq and lifetime (#5191) (Nikolas Pitsillos)
    • [c17e6416] - improve warning about config file in custom docker (#5190) (Joan Fontanals)
    • [22330fcb] - clarify return behavior in parameters (#5179) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [b5977f47] - update contributing to remove labeling (#5225) (Joan Fontanals)
    • [45c7c6e1] - update pull request template link (#5214) (Joan Fontanals)
    • [1eab9ce6] - fix doc template (Han Xiao)
    • [8c2d0758] - docs: update TOC (Jina Dev Bot)
    • [139b068e] - version: the next version will be 3.9.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.2(Sep 15, 2022)

    Release Note (3.9.2)

    Release time: 2022-09-15 14:00:36

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Girish Chandrashekar, zhangkai, AlaeddineAbdessalem, Joan Fontanals, Jina Dev Bot, Andrei Ungureanu, Deepankar Mahapatro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [c51f9014] - hubble async push (#5129) (zhangkai)
    • [3f39ed46] - runtime: add argument to specify exceptions that will exit the runtime (#5165) (Girish Chandrashekar)
    • [27e1f779] - hubio: display warning messages from hubble request (jina hub push) (#5156) (Andrei Ungureanu)

    ๐Ÿž Bug fixes

    • [29ad1750] - exit on exception args only applies to executors (#5169) (AlaeddineAbdessalem)
    • [55188165] - pin docarray version for new column syntax (#5171) (Joan Fontanals)
    • [fa83955c] - increase minimum protobuf version (#5166) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [28aeac8e] - add section for exit_on_exceptions argument (#5172) (Girish Chandrashekar)
    • [33891c46] - jcloud: labels in flow yaml (#5164) (Deepankar Mahapatro)

    ๐Ÿ Unit Test and CICD

    • [b59d0450] - add jina auth token (#5167) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [76dca020] - update announcement in readme and docs (Jina Dev Bot)
    • [f4248ea8] - docs: update TOC (Jina Dev Bot)
    • [fda5623a] - version: the next version will be 3.9.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.1(Sep 8, 2022)

    Release Note (3.9.1)

    Release time: 2022-09-08 14:10:44

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [273fda5a] - merge dryrun into ping (#5151) (Han Xiao)

    ๐Ÿน Other Improvements

    • [0a71009b] - version: the next version will be 3.9.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.0(Sep 8, 2022)

    Highlights :star2:

    Update to new protobuf version

    We have updated our protobuf dependency to use the latest protobuf version leveraging all the performance improvement that comes with it.

    #5082

    Merge manifest.yml and config.yml for Jina Hub Executors

    We have made it easier for the users to manage their Hub Executors by merging the information from manifest.yml to config.yml which makes the experience much better.

    #5101

    Expose grpc server parameters to Executor

    We expose all the parameters that grpc server can accept for users to be able to configure their services in the most robust and reliable way.

    #5092

    Other changes

    • Add more Dockerfile templates options in jina hub new #5104
    • Add failed and successfull requests metrics #5079

    Bug fixes

    • Fix reading empty list from environment variables #5143
    Source code(tar.gz)
    Source code(zip)
  • v3.8.4(Sep 8, 2022)

    Release Note (3.8.4)

    Release time: 2022-09-08 08:51:32

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, AlaeddineAbdessalem, Joan Fontanals, Johannes Messner, Andrei Ungureanu, samsja, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [d1faf6e6] - pass internal flag to telemetry (#5134) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [e501ddfd] - version: adapt to docker hub v2 (#5146) (Deepankar Mahapatro)
    • [d76e69ac] - fix parsing empty list (#5143) (AlaeddineAbdessalem)
    • [3ec33c08] - gpu dockerfile template include now jina install (#5124) (samsja)
    • [e2967f98] - set minimal docarray dependency (#5133) (Joan Fontanals)
    • [ff803686] - linkerd cd (#5131) (AlaeddineAbdessalem)

    ๐Ÿงผ Code Refactoring

    • [a78612f3] - manifest config (#5101) (Andrei Ungureanu)

    ๐Ÿ Unit Test and CICD

    • [ddf90efb] - remove outdated test (#5139) (Johannes Messner)
    • [a4ea7215] - increase wait time (#5137) (AlaeddineAbdessalem)
    • [478e53f6] - fix k8s tests return responses params (#5127) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a9b80aec] - docs: update TOC (Jina Dev Bot)
    • [1df9b459] - version: the next version will be 3.8.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.3(Sep 5, 2022)

    Release Note (3.8.3)

    Release time: 2022-09-05 14:48:30

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [7b7eecc5] - cap grpc version (#5122) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [47e2cfc3] - docs: update TOC (Jina Dev Bot)
    • [0dbdcaa0] - version: the next version will be 3.8.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.2(Sep 3, 2022)

    Release Note (3.8.2)

    Release time: 2022-09-03 15:56:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Zac Li, samsja, AlaeddineAbdessalem, Jina Dev Bot, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [e794c06c] - expose grpc parameters and add production ready keepalive parameters (#5092) (samsja)
    • [6205ffc9] - add gpu dockerfile support to jina hub new (#5104) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [9293698b] - rewrite flow add (Han Xiao)
    • [115bc0fa] - rewrite flow add (#5120) (Han Xiao)
    • [22fdbcee] - jcloud: fix jc status link (#5116) (Zac Li)
    • [4dfe6dfb] - jcloud: fix docs for gateway resource customization (#5114) (Zac Li)
    • [6b24bf69] - correct jcloud retention days (#5109) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [587f6268] - fix test concurrent async flow (#5115) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [763f6773] - update readme (Han Xiao)
    • [2651c506] - disable hub-integration (Han Xiao)
    • [3590c385] - fix readme (Han Xiao)
    • [8303fb25] - docs: improve docs on jcloud k8s (Han Xiao)
    • [a34edc62] - update announcement in readme and docs (Jina Dev Bot)
    • [d124d9af] - docs: update TOC (Jina Dev Bot)
    • [0df7cd8a] - version: the next version will be 3.8.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.1(Aug 30, 2022)

    Release Note (3.8.1)

    Release time: 2022-08-30 19:32:52

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6df36ce5] - client: add profiling function to client and flow (#5105) (Han Xiao)

    ๐Ÿž Bug fixes

    • [78aae025] - fix inconsistent docs and tests on py_modules (#5108) (Han Xiao)
    • [a725010d] - remove flakyness of monitoring integration tests in CI (#5106) (samsja)

    ๐Ÿงผ Code Refactoring

    • [90fa81f3] - move port monitoring to pods parsing (#5100) (samsja)

    ๐Ÿ“— Documentation

    • [95ee283c] - fix naming (#5102) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [d528d947] - fix cd pipeline (Han Xiao)
    • [57625350] - docs: update TOC (Jina Dev Bot)
    • [1ad06216] - docs: fix retention days description (Han Xiao)
    • [9f5f5879] - version: the next version will be 3.8.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.0(Aug 29, 2022)

    Release Note (3.8.0)

    Release time: 2022-08-29 16:43:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, tarrantro, Han Xiao, AlaeddineAbdessalem, samsja, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [527beb85] - bump protobuf version (#5082) (AlaeddineAbdessalem)
    • [c47cb716] - add failed and successful request number metrics (#5079) (AlaeddineAbdessalem)

    ๐Ÿž Bug fixes

    • [c81252ac] - update to protobuf 4.21 new types (#5098) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [6e51e815] - refactor data request handler monitoring (#5088) (samsja)

    ๐Ÿ“— Documentation

    • [b3fdb49c] - correct jcloud retention days (#5096) (tarrantro)
    • [98fb71f1] - jcloud: adjust jcloud monitoring docs(#5081) (Zac Li)
    • [dbea43f3] - remove kong/alb from jcloud document (#5080) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [27ec3b41] - cleanup pip install (#5094) (AlaeddineAbdessalem)
    • [c0c3bf9a] - downgrade linkerd version in CI (#5090) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [142c0bb1] - docs: add streamline value props (Han Xiao)
    • [af7f6763] - fix readme (Han Xiao)
    • [c39714fa] - docs: add install apple silicon (Han Xiao)
    • [5375b4d3] - update announcement in readme and docs (Jina Dev Bot)
    • [53553b14] - add security report (Han Xiao)
    • [cfe5dd70] - update readme (Han Xiao)
    • [e106796c] - docs: update furo deps (Han Xiao)
    • [079d04b0] - docs: add jcloud to first app (Han Xiao)
    • [64de8608] - docs: add comparing alternatives (Han Xiao)
    • [f9c2d8de] - docs: update TOC (Jina Dev Bot)
    • [2154f98f] - version: the next version will be 3.7.15 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.14(Aug 22, 2022)

    Release Note (3.7.14)

    Release time: 2022-08-22 13:52:20

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿน Other Improvements

    • [4f380c7e] - fix pprint (Han Xiao)
    • [0bd289fe] - docs: update TOC (Jina Dev Bot)
    • [96489914] - version: the next version will be 3.7.14 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.13(Aug 21, 2022)

    Release Note (3.7.13)

    Release time: 2022-08-21 21:03:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b7bf0ee1] - set workspace in executor init method (#5072) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [f6dcde38] - telemetry (#5078) (Han Xiao)

    ๐Ÿน Other Improvements

    • [8ad0ead5] - docs: refactor create project docs (Han Xiao)
    • [c7e19841] - update readme (Han Xiao)
    • [7b034d34] - docs: refactor telemetry docs (Han Xiao)
    • [76f48f8f] - docs: refactor jcloud docs (Han Xiao)
    • [e506da44] - docs: refactor hub docs (Han Xiao)
    • [b91f2084] - docs: refactor executors docs (Han Xiao)
    • [f8a63ccf] - docs: refactor flow basic docs (Han Xiao)
    • [885914ad] - docs: restructure docs (Han Xiao)
    • [36fdebe2] - docs: fix docs in jcloud (Han Xiao)
    • [4673e7ab] - docs: update TOC (Jina Dev Bot)
    • [a4c48d1b] - version: the next version will be 3.7.13 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.12(Aug 18, 2022)

    Release Note (3.7.12)

    Release time: 2022-08-18 22:23:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6f87524c] - include jcloud cli (#5074) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [10c63db2] - update pr template with reminder to add a description (#5062) (Johannes Messner)
    • [6fbd2be9] - docs: update TOC (Jina Dev Bot)
    • [7d82fdc3] - version: the next version will be 3.7.12 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.11(Aug 17, 2022)

    Release Note (3.7.11)

    Release time: 2022-08-17 21:09:48

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [15fd60d5] - cli: jina auth cli error (Han Xiao)

    ๐Ÿน Other Improvements

    • [80393316] - version: the next version will be 3.7.11 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.10(Aug 17, 2022)

    Release Note (3.7.10)

    Release time: 2022-08-17 12:22:49

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b9bddd22] - cli: jina auth cli error (#5070) (Han Xiao)

    ๐Ÿ“— Documentation

    • [840c221f] - add gateway timeout and update shared gpu (#5069) (tarrantro)

    ๐Ÿน Other Improvements

    • [bffe263c] - docs: update TOC (Jina Dev Bot)
    • [94c83cdb] - version: the next version will be 3.7.10 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.9(Aug 16, 2022)

    Release Note (3.7.9)

    Release time: 2022-08-16 16:25:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    ๐Ÿงผ Code Refactoring

    • [c1c500aa] - remove unused hub environment variables (#5067) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [442a86cd] - version: the next version will be 3.7.9 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.8(Aug 16, 2022)

    Release Note (3.7.8)

    Release time: 2022-08-16 15:18:50

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [bf864748] - hub: use hubble sdk (#5064) (Han Xiao)

    ๐Ÿ“— Documentation

    • [317e9ce1] - jcloud: doc for gateway customization (#5065) (Zac Li)

    ๐Ÿน Other Improvements

    • [a6be9481] - update announcement in readme and docs (Jina Dev Bot)
    • [f3bcdaa8] - docs: update TOC (Jina Dev Bot)
    • [dfd1e07d] - docs: fix typos (Han Xiao)
    • [d579cb63] - version: the next version will be 3.7.8 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.7(Aug 15, 2022)

    Release Note (3.7.7)

    Release time: 2022-08-15 18:14:16

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [22563074] - cache hub pull with build env (#5061) (Johannes Messner)

    ๐Ÿน Other Improvements

    • [e7377947] - docs: fix typos (Han Xiao)
    • [6d218fbf] - version: the next version will be 3.7.7 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.6(Aug 15, 2022)

    Release Note (3.7.6)

    Release time: 2022-08-15 14:04:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [49a842d9] - unify temp path to dot cache jina (#5058) (Han Xiao)

    ๐Ÿ“— Documentation

    • [731d492c] - add CPU and GPU type (#5060) (tarrantro)
    • [d8f6f418] - rewrite the telemetry session (#5059) (Han Xiao)
    • [01111539] - jcloud: autoscale docs (#5056) (Zac Li)

    ๐Ÿน Other Improvements

    • [299a908b] - docs: add what is jina (Han Xiao)
    • [23914a81] - add what is cross multi modal (Han Xiao)
    • [0c78b993] - docs: update TOC (Jina Dev Bot)
    • [102bc6de] - version: the next version will be 3.7.6 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.5(Aug 10, 2022)

    Release Note (3.7.5)

    Release time: 2022-08-10 10:19:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Jina Dev Bot, zhangkai, samsja, Deepankar Mahapatro, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6cc4497a] - hubble support env variables (#5023) (zhangkai)
    • [12404535] - add dryrun to cli (#5050) (samsja)

    ๐Ÿ“— Documentation

    • [1d60655d] - jcloud: fix formatting (#5048) (Deepankar Mahapatro)
    • [6df83cf1] - improve grpc third party client documentation (#5047) (samsja)
    • [8c6321ac] - refactor resources, add gpu, kong (#5027) (tarrantro)

    ๐Ÿน Other Improvements

    • [d6d4c30a] - update announcement in readme and docs (Jina Dev Bot)
    • [9fee2dff] - docs: update TOC (Jina Dev Bot)
    • [fe7b8894] - version: the next version will be 3.7.5 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.4(Aug 3, 2022)

    Release Note (3.7.4)

    Release time: 2022-08-03 12:31:57

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [4c3d760f] - do not send target_executor to Executors (#5041) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a51b42cc] - update announcement in readme and docs (Jina Dev Bot)
    • [ecb95829] - docs: update TOC (Jina Dev Bot)
    • [286bdce8] - version: the next version will be 3.7.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.3(Jul 28, 2022)

    Release Note (3.7.3)

    Release time: 2022-07-28 14:29:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [032bd5e6] - fix specific params problem with branches (#5038) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [81999e97] - version: the next version will be 3.7.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.2(Jul 28, 2022)

    Release Note (3.7.2)

    Release time: 2022-07-28 09:19:06

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [fb88015e] - add more logging in grpc dry run (#5036) (Zac Li)

    ๐Ÿน Other Improvements

    • [5116b922] - docs: update TOC (Jina Dev Bot)
    • [32dd47af] - version: the next version will be 3.7.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

txtai executes machine-learning workflows to transform data and build AI-powered semantic search applications.

NeuML 3.1k Dec 31, 2022
Image search service based on imgsmlr extension of PostgreSQL. Support image search by image.

imgsmlr-server Image search service based on imgsmlr extension of PostgreSQL. Support image search by image. This is a sample application of imgsmlr.

jie 45 Dec 12, 2022
Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Wagtail CLIP allows you to search your Wagtail images using natural language queries.

Matt Segal 10 Dec 21, 2022
rclip - AI-Powered Command-Line Photo Search Tool

rclip is a command-line photo search tool based on the awesome OpenAI's CLIP neural network.

Yurij Mikhalevich 394 Dec 12, 2022
Deep Image Search - AI-Based Image Search Engine

Deep Image Search is an AI-based image search engine that includes deep transfer learning features Extraction and tree-based vectorized search technique.

null 144 Jan 5, 2023
Search emails from a domain through search engines

EmailFinder - search emails through Search Engines

Josuรฉ Encinar 155 Dec 30, 2022
GitScanner is a script to make it easy to search for Exposed Git through an advanced Google search.

GitScanner Legal disclaimer Usage of GitScanner for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to

Kaio Gomes 3 Oct 28, 2022
A fast, efficiency python package for searching and getting search results with many different search engines

search A fast, efficiency python package for searching and getting search results with many different search engines. Installation To install the pack

Neurs 0 Oct 6, 2022
Super Simple Similarities Service

Super Simple Similarities Service

vincent d warmerdam 95 Dec 25, 2022
Modular search for Django

Haystack Author: Daniel Lindsley Date: 2013/07/28 Haystack provides modular search for Django. It features a unified, familiar API that allows you to

Haystack Search 3.4k Jan 4, 2023
Full text search for flask.

flask-msearch Installation To install flask-msearch: pip install flask-msearch # when MSEARCH_BACKEND = "whoosh" pip install whoosh blinker # when MSE

honmaple 197 Dec 29, 2022
Senginta is All in one Search Engine Scrapper for used by API or Python Module. It's Free!

Senginta is All in one Search Engine Scrapper. With traditional scrapping, Senginta can be powerful to get result from any Search Engine, and convert to Json. Now support only for Google Product Search Engine (GShop, GVideo and many too) and Baidu Search Engine.

null 33 Nov 21, 2022
document organizer with tags and full-text-search, in a simple and clean sqlite3 schema

document organizer with tags and full-text-search, in a simple and clean sqlite3 schema

Manos Pitsidianakis 152 Oct 29, 2022
Google Search Engine Results Pages (SERP) in locally, no API key, no signup required

Local SERP Google Search Engine Results Pages (SERP) in locally, no API key, no signup required Make sure the chromedriver and required package are in

theblackcat102 4 Jun 29, 2021
A web search server for ParlAI, including Blenderbot2.

Description A web search server for ParlAI, including Blenderbot2. Querying the server: The server reacting correctly: Uses html2text to strip the mar

Jules Gagnon-Marchand 119 Jan 6, 2023
This project is a sample demo of Arxiv search related to AI/ML Papers built using Streamlit, sentence-transformers and Faiss.

This project is a sample demo of Arxiv search related to AI/ML Papers built using Streamlit, sentence-transformers and Faiss.

Karn Deb 49 Oct 30, 2022
Google Project: Search and auto-complete sentences within given input text files, manipulating data with complex data-structures.

Auto-Complete Google Project In this project there is an implementation for one feature of Google's search engines - AutoComplete. Autocomplete, or wo

Hadassah Engel 10 Jun 20, 2022
Full-text multi-table search application for Django. Easy to install and use, with good performance.

django-watson django-watson is a fast multi-model full-text search plugin for Django. It is easy to install and use, and provides high quality search

Dave Hall 1.1k Jan 3, 2023
An image inline search telegram bot.

Image-Search-Bot An image inline search telegram bot. Note: Use Telegram picture bot. That is better. Not recommending to deploy this bot. Made with P

Fayas Noushad 24 Oct 21, 2022