An easier way to build neural search on the cloud

Last update: May 27, 2022

Overview

An easier way to build neural search on the cloud

Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g. text, images, video, audio) on the cloud.

⏱️ Time Saver - The design pattern of neural search systems, from zero to a production-ready system in minutes.

🍱 Full-Stack Ownership - Keep an end-to-end stack ownership of your solution, avoid the integration pitfalls with fragmented, multi-vendor, generic legacy tools.

🌌 Universal Search - Large-scale indexing and querying of unstructured data: video, image, long/short text, music, source code, etc.

🧠 First-Class AI Models - First-class support for state-of-the-art AI models, easily usable and extendable with a Pythonic interface.

🌩️ Fast & Cloud Ready - Decentralized architecture from day one. Scalabe & cloud-native by design: enjoy containerizing, distributing, sharding, async, REST/gRPC/WebSocket.

❤️ Made with Love - Never compromise on quality, actively maintained by a passionate full-time, venture-backed team.

Docs • Hello World • Quick Start • Learn • Examples • Contribute • Jobs • Website • Slack

Installation

📦 _{^{x86/64,arm/v6,v7,v8 (Apple M1)}}	On Linux/macOS & Python 3.7/3.8/3.9	Docker Users
Standard	`pip install -U jina`	`docker run jinaai/jina:latest`
_Daemon	_{pip install -U "jina[daemon]"}	_{docker run --network=host jinaai/jina:latest-daemon}
_{With Extras}	_{pip install -U "jina[devel]"}	_{docker run jinaai/jina:latest-devel}
_{Dev/Pre-Release}	_{pip install --pre jina}	_{docker run jinaai/jina:master}

Version identifiers are explained here. To install Jina with extra dependencies please refer to the docs. Jina can run on Windows Subsystem for Linux. We welcome the community to help us with native Windows support.

Jina "Hello, World!" 👋 🌍

Just starting out? Try Jina's "Hello, World" - a simple image neural search demo for Fashion-MNIST. No extra dependencies needed, simply run:

jina hello-world  # more options in --help

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html
# replace "open" with "xdg-open" on Linux

Click here to see console output

This downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and after running opens a webpage and shows results:

Covid-19 Chatbot

For NLP engineers, we provide a simple chatbot demo for answering Covid-19 questions. You will need PyTorch and Transformers, which can be installed along with Jina:

pip install "jina[torch,transformers]"
jina hello-world-chatbot

This downloads CovidQA dataset and tells Jina to index 418 question-answer pairs with DistilBERT. The index process takes about 1 minute on CPU. Then it opens a webpage where you can input questions and ask Jina.

Get Started


🥚	CRUD Functions • Document • Flow
🐣	Feed Data • Fetch Result • Add Logic • Inter & Intra Parallelism • Decentralize • Asynchronous
🐥	Customize Encoder • Test Encoder • Parallelism & Batching • Add Data Indexer • Compose Flow from YAML • Search • Evaluation • REST Interface

🥚 Fundamental

CRUD Functions

First we look at basic CRUD operations. In Jina, CRUD corresponds to four functions: index (create), search (read), update, and delete. With Documents below as an example:

import numpy as np
from jina import Document
docs = [Document(id='🐲', embedding=np.array([0, 0]), tags={'guardian': 'Azure Dragon', 'position': 'East'}),
        Document(id='🐦', embedding=np.array([1, 0]), tags={'guardian': 'Vermilion Bird', 'position': 'South'}),
        Document(id='🐢', embedding=np.array([0, 1]), tags={'guardian': 'Black Tortoise', 'position': 'North'}),
        Document(id='🐯', embedding=np.array([1, 1]), tags={'guardian': 'White Tiger', 'position': 'West'})]

Let's build a Flow with a simple indexer:

from jina import Flow
f = Flow().add(uses='_index')

Document and Flow are basic concepts in Jina, which will be explained later. _index is a built-in embedding + structured storage that one can use out of the box.

Index	# save four docs (both embedding and structured info) into storage with f: f.index(docs, on_done=print)
Search	# retrieve top-3 neighbours of 🐲, this print 🐲🐦🐢 with score 0, 1, 1 respectively with f: f.search(docs[0], top_k=3, on_done=lambda x: print(x.docs[0].matches)) ^{{"id": "🐲", "tags": {"guardian": "Azure Dragon", "position": "East"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1} {"id": "🐦", "tags": {"position": "South", "guardian": "Vermilion Bird"}, "embedding": {"dense": {"buffer": "AQAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1} {"id": "🐢", "tags": {"guardian": "Black Tortoise", "position": "North"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAABAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1}}
Update	# update 🐲 embedding in the storage docs[0].embedding = np.array([1, 1]) with f: f.update(docs[0])
Delete	# remove 🐦🐲 Documents from the storage with f: f.delete(['🐦', '🐲'])

Document

Document is Jina's primitive data type. It can contain text, image, array, embedding, URI, and accompanied by rich meta information. To construct a Document, one can use:

import numpy
from jina import Document

doc1 = Document(content=text_from_file, mime_type='text/x-python')  # a text document contains python code
doc2 = Document(content=numpy.random.random([10, 10]))  # a ndarray document

Document can be recurred both vertically and horizontally to have nested documents and matched documents. To better see the recursive structure of a document, one can use .plot() function. If you are using JupyterLab/Notebook, all Document objects will be auto-rendered.

import numpy
from jina import Document

d0 = Document(id='🐲', embedding=np.array([0, 0]))
d1 = Document(id='🐦', embedding=np.array([1, 0]))
d2 = Document(id='🐢', embedding=np.array([0, 1]))
d3 = Document(id='🐯', embedding=np.array([1, 1]))

d0.chunks.append(d1)
d0.chunks[0].chunks.append(d2)
d0.matches.append(d3)

d0.plot()  # simply `d0` on JupyterLab

Click here to see more about MultimodalDocument

MultimodalDocument

A MultimodalDocument is a document composed of multiple Document from different modalities (e.g. text, image, audio).

Jina provides multiple ways to build a multimodal Document. For example, one can provide the modality names and the content in a dict:

from jina import MultimodalDocument
document = MultimodalDocument(modality_content_map={
    'title': 'my holiday picture',
    'description': 'the family having fun on the beach',
    'image': PIL.Image.open('path/to/image.jpg')
})

One can also compose a MultimodalDocument from multiple Document directly:

from jina.types import Document, MultimodalDocument

doc_title = Document(content='my holiday picture', modality='title')
doc_desc = Document(content='the family having fun on the beach', modality='description')
doc_img = Document(content=PIL.Image.open('path/to/image.jpg'), modality='image')
doc_img.tags['date'] = '10/08/2019'

document = MultimodalDocument(chunks=[doc_title, doc_description, doc_img])

Fusion Embeddings from Different Modalities

To extract fusion embeddings from different modalities Jina provides BaseMultiModalEncoder abstract class, which has a unqiue encode interface.

def encode(self, *data: 'numpy.ndarray', **kwargs) -> 'numpy.ndarray':
    ...

MultimodalDriver provides data to the MultimodalDocument in the correct expected order. In this example below, image embedding is passed to the endoder as the first argument, and text as the second.

!MyMultimodalEncoder
with:
  positional_modality: ['image', 'text']
requests:
  on:
    [IndexRequest, SearchRequest]:
      - !MultiModalDriver {}

Interested readers can refer to jina-ai/example: how to build a multimodal search engine for image retrieval using TIRG (Composing Text and Image for Image Retrieval) for the usage of MultimodalDriver and BaseMultiModalEncoder in practice.

Flow

Jina provides a high-level Flow API to simplify building CRUD workflows. To create a new Flow:

from jina import Flow
f = Flow().add()

This creates a simple Flow with one Pod. You can chain multiple .add()s in a single Flow.

To visualize the Flow, simply chain it with .plot('my-flow.svg'). If you are using a Jupyter notebook, the Flow object will be displayed inline without plot.

Gateway is the entrypoint of the Flow.

Get the vibe? Now we are talking! Let's learn more about the basic concepts and features in Jina.


🥚	CRUD Functions • Document • Flow
🐣	Feed Data • Fetch Result • Add Logic • Inter & Intra Parallelism • Decentralize • Asynchronous
🐥	Customize Encoder • Test Encoder • Parallelism & Batching • Add Data Indexer • Compose Flow from YAML • Search • Evaluation • REST Interface

🐣 Basic

Feed Data

To use a Flow, open it via with context manager, like you would open a file in Python. Now let's create some empty document and index it:

from jina import Document

with Flow().add() as f:
    f.index((Document() for _ in range(10)))

Flow supports CRUD operations: index, search, update, delete. Besides, it also provides sugary syntax on ndarray, csv, ndjson and arbitrary files.

Input	Example on `index`/`search`	Explain
`numpy.ndarray`	^{with f: f.index_ndarray(numpy.random.random([4,2]))}	Input four `Document`, each `document.blob` is a `ndarray([2])`
CSV	^{with f, open('index.csv') as fp: f.index_csv(fp1, field_resolver={'pic_url': 'uri'})}	Each line in the `index.csv` is constructed as `Document`, CSV's field `pic_url` is mapped to `document.uri`.
JSON Lines/`ndjson`/LDJSON	^{with f, open('index.ndjson') as fp: f.index_ndjson(fp1, field_resolver={'question_id': 'id'})}	Each line in `index.ndjson` is constructed as `Document`, JSON's field `question_id` is mapped to `document.id`.
Files with wildcard	^{with f: f.index_files(['/tmp/.mp4', '/tmp/.pdf'])}	Each file captured is constructed as a `Document`, whose content (`text`, `blob`, `buffer`) is auto-guessed & filled.

Fetch Result

Once a request is done, callback functions are fired. Jina Flow implements Promise-like interface, you can add callback functions on_done, on_error, on_always to hook different events. In the example below, our Flow passes the message then prints the result when successful. If something wrong, it beeps. Finally, the result is written to output.txt.

def beep(*args):
    # make a beep sound
    import os
    os.system('echo -n "\a";')

with Flow().add() as f, open('output.txt', 'w') as fp:
    f.index(numpy.random.random([4, 5, 2]),
            on_done=print, on_error=beep, on_always=lambda x: fp.write(x.json()))

Add Logic

To add logic to the Flow, use the uses parameter to attach a Pod with an Executor. uses accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.

f = (Flow().add(uses='MyBertEncoder')  # class name of a Jina Executor
           .add(uses='docker://jinahub/pod.encoder.dummy_mwu_encoder:0.0.6-0.9.3')  # the image name
           .add(uses='myencoder.yml')  # YAML serialization of a Jina Executor
           .add(uses='!WaveletTransformer | {freq: 20}')  # inline YAML config
           .add(uses='_pass')  # built-in shortcut executor
           .add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}}))  # dict config object with __cls keyword

The power of Jina lies in its decentralized architecture: each add creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.

Inter & Intra Parallelism

Chaining .add()s creates a sequential Flow. For parallelism, use the needs parameter:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', needs='gateway')
           .needs(['p1','p2', 'p3'], name='r1').plot())

p1, p2, p3 now subscribe to Gateway and conduct their work in parallel. The last .needs() blocks all Pods until they finish their work. Note: parallelism can also be performed inside a Pod using parallel:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', parallel=3)
           .needs(['p1','p3'], name='r1').plot())

Decentralized Flow

A Flow does not have to be local-only, one can put any Pod to remote(s). In the example below, with the host keyword gpu-pod is put to a remote machine for parallelization, whereas other pods stay local. Extra file dependencies that need to be uploaded are specified via the upload_files keyword.

123.456.78.9

# have docker installed
docker run --name=jinad --network=host -v /var/run/docker.sock:/var/run/docker.sock jinaai/jina:latest-daemon --port-expose 8000
# to stop it
docker rm -f jinad

Local

import numpy as np
from jina import Flow

f = (Flow()
     .add()
     .add(name='gpu_pod',
          uses='mwu_encoder.yml',
          host='123.456.78.9:8000',
          parallel=2,
          upload_files=['mwu_encoder.py'])
     .add())

with f:
    f.index_ndarray(np.random.random([10, 100]), output=print)

We provide a demo server on cloud.jina.ai:8000, give the following snippet a try!

from jina import Flow

with Flow().add().add(host='cloud.jina.ai:8000') as f:
    f.index(['hello', 'world'])

Asynchronous Flow

Synchronous from outside, Jina runs asynchronously underneath: it manages the eventloop(s) for scheduling the jobs. If the user wants more control over the eventloop, then AsyncFlow comes to use.

Unlike Flow, the CRUD of AsyncFlow accepts input & output functions as async generator. This is useful when your data sources involve other asynchronous libraries (e.g. motor for MongoDB):

from jina import AsyncFlow

async def input_fn():
    for _ in range(10):
        yield Document()
        await asyncio.sleep(0.1)

with AsyncFlow().add() as f:
    async for resp in f.index(input_fn):
        print(resp)

AsyncFlow is particular useful when Jina is using as part of the integration, where another heavy-lifting job is running concurrently:

async def run_async_flow_5s():  # WaitDriver pause 5s makes total roundtrip ~5s
    with AsyncFlow().add(uses='- !WaitDriver {}') as f:
        async for resp in f.index_ndarray(numpy.random.random([5, 4])):
            print(resp)

async def heavylifting():  # total roundtrip takes ~5s
    print('heavylifting other io-bound jobs, e.g. download, upload, file io')
    await asyncio.sleep(5)
    print('heavylifting done after 5s')

async def concurrent_main():  # about 5s; but some dispatch cost, can't be just 5s, usually at <7s
    await asyncio.gather(run_async_flow_5s(), heavylifting())

if __name__ == '__main__':
    asyncio.run(concurrent_main())

AsyncFlow is very useful when using Jina inside the Jupyter Notebook. As Jupyter/ipython already manages an eventloop and thanks to autoawait, AsyncFlow can run out-of-the-box in Jupyter.

That's all you need to know for understanding the magic behind hello-world. Now let's dive into it!


🥚	CRUD Functions • Document • Flow
🐣	Feed Data • Fetch Result • Add Logic • Inter & Intra Parallelism • Decentralize • Asynchronous
🐥	Customize Encoder • Test Encoder • Parallelism & Batching • Add Data Indexer • Compose Flow from YAML • Search • Evaluation • REST Interface

🐥 Breakdown of `hello-world`

Customize Encoder

Let's first build a naive image encoder that embeds images into vectors using an orthogonal projection. To do this, we simply inherit from BaseImageEncoder: a base class from the jina.executors.encoders module. We then override its __init__() and encode() methods.

import numpy as np
from jina.executors.encoders import BaseImageEncoder

class MyEncoder(BaseImageEncoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        np.random.seed(1337)
        H = np.random.rand(784, 64)
        u, s, vh = np.linalg.svd(H, full_matrices=False)
        self.oth_mat = u @ vh

    def encode(self, data: 'np.ndarray', *args, **kwargs):
        return (data.reshape([-1, 784]) / 255) @ self.oth_mat

Jina provides a family of Executor classes, which summarize frequently-used algorithmic components in neural search. This family consists of encoders, indexers, crafters, evaluators, and classifiers, each with a well-designed interface. You can find the list of all 107 built-in executors here. If they don't meet your needs, inheriting from one of them is the easiest way to bootstrap your own Executor. Simply use our Jina Hub CLI:

pip install jina[hub] && jina hub new

Test Encoder in Flow

Let's test our encoder in the Flow with some synthetic data:

def validate(req):
    assert len(req.docs) == 100
    assert NdArray(req.docs[0].embedding).value.shape == (64,)

f = Flow().add(uses='MyEncoder')

with f:
    f.index_ndarray(numpy.random.random([100, 28, 28]), on_done=validate)

All good! Now our validate function confirms that all one hundred 28x28 synthetic images have been embedded into 100x64 vectors.

Parallelism & Batching

By setting a larger input, you can play with batch_size and parallel:

f = Flow().add(uses='MyEncoder', parallel=10)

with f:
    f.index_ndarray(numpy.random.random([60000, 28, 28]), batch_size=1024)

Add Data Indexer

Now we need to add an indexer to store all the embeddings and the image for later retrieval. Jina provides a simple numpy-powered vector indexer NumpyIndexer, and a key-value indexer BinaryPbIndexer. We can combine them in a single YAML file:

!CompoundIndexer
components:
  - !NumpyIndexer
    with:
      index_filename: vec.gz
  - !BinaryPbIndexer
    with:
      index_filename: chunk.gz
metas:
  workspace: ./

! tags a structure with a class name
with defines arguments for initializing this class object.

Essentially, the above YAML config is equivalent to the following Python code:

from jina.executors.indexers.vector import NumpyIndexer
from jina.executors.indexers.keyvalue import BinaryPbIndexer
from jina.executors.indexers import CompoundIndexer

a = NumpyIndexer(index_filename='vec.gz')
b = BinaryPbIndexer(index_filename='vec.gz')
c = CompoundIndexer()
c.components = lambda: [a, b]

Compose Flow from YAML

Now let's add our indexer YAML file to the Flow with .add(uses=). Let's also add two shards to the indexer to improve its scalability:

f = Flow().add(uses='MyEncoder', parallel=2).add(uses='myindexer.yml', shards=2).plot()

When you have many arguments, constructing a Flow in Python can get cumbersome. In that case, you can simply move all arguments into one flow.yml:

!Flow
version: '1.0'
pods:
  - name: encode
    uses: MyEncoder
    parallel: 2
  - name:index
    uses: myindexer.yml
    shards: 2

And then load it in Python:

f = Flow.load_config('flow.yml')

Search

Querying a Flow is similar to what we did with indexing. Simply load the query Flow and switch from f.index to f.search. Say you want to retrieve the top 50 documents that are similar to your query and then plot them in HTML:

f = Flow.load_config('flows/query.yml')
with f:
    f.search_ndarray(numpy.random.random([10, 28, 28]), shuffle=True, on_done=plot_in_html, top_k=50)

Evaluation

To compute precision recall on the retrieved result, you can add _eval_pr, a built-in evaluator for computing precision & recall.

f = (Flow().add(...)
           .add(uses='_eval_pr'))

You can construct an iterator of query and groundtruth pairs and feed to the flow f, via:

from jina import Document

def query_generator():
    for _ in range(10):
        q = Document()
        # now construct expect matches as groundtruth
        gt = Document(q, copy=True)  # make sure 'gt' is identical to 'q'
        gt.matches.append(...)
        yield q, gt

f.search(query_iterator, ...)

REST Interface

In practice, the query Flow and the client (i.e. data sender) are often physically separated. Moreover, the client may prefer to use a REST API rather than gRPC when querying. You can set port_expose to a public port and turn on REST support with restful=True:

f = Flow(port_expose=45678, restful=True)

with f:
    f.block()

That is the essence behind jina hello-world. It is merely a taste of what Jina can do. We’re really excited to see what you do with Jina! You can easily create a Jina project from templates with one terminal command:

pip install jina[hub] && jina hub new --type app

This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Learn

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited

Jina 101: First Things to Learn About Jina

English • 日本語 • Français • Português • Deutsch • Русский язык • 中文 • عربية

Examples (View all)

Example code to build your own projects

📄	Semantic Wikipedia Search with Transformers and DistilBERT Brand new to neural search? See a simple text-search example to understand how Jina works
📄	Add Incremental Indexing to Wikipedia Search Index more effectively by adding incremental indexing to your Wikipedia search
📄	Build a NLP Semantic Search System with Transformers Upgrade from plain search to sentence search and practice your Flows and Pods by searching sentences from Wikipedia
📄	Search Lyrics with Transformers and PyTorch Get a better understanding of chunks by searching a lyrics database. Now with shiny front-end!
🖼️	Google's Big Transfer Model in (Poké-)Production Use SOTA visual representation for searching Pokémon!
🎧	Search YouTube audio data with Vggish A demo of neural search for audio data based Vggish model.
🎞️	Search Tumblr GIFs with KerasEncoder Use prefetching and sharding to improve the performance of your index and query flow when searching animated GIFs.

Please check our examples repo for advanced and community-submitted examples.

Want to read more? Check our Founder Han Xiao's blog and our official blog.

Documentation

Apart from the learning resources we provided above, We highly recommended you go through our documentation to master Jina.

Our docs are built on every push, merge, and release of Jina's master branch. Documentation for older versions is archived here.

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors ✨

Community

Code of conduct - play nicely with the Jina community
Slack workspace - join #general on our Slack to meet the team and ask questions
YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
LinkedIn - get to know Jina AI as a company and find job opportunities
- follow and interact with us using hashtag #JinaSearch
Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Issues

Text-by-Audio

Hi,

I want to retrieve text by searching for an audio using AudioClip model.

First, I created indexing of text (car-horn, coughing, alarm-clock, thunderstorm etc) and using AudioClipTextEncoder for embedding.

After that I am searching for an audio where i am using AudioClipEncoder for embedding.

For both text and audio indexing simple-indexer is used.

While searching for an audio i created chunks using segmenter and using MyRanker for ranking scores where i modified some script.

The output is: input_audio: AudioCLIP/demo/audio/thunder_3-144891-B-19.wav, len-of-chunks:1 +- id: e5eceb26737311ec83fd2fdc0fc83b8e score: 0.931110, cat input_audio: AudioCLIP/demo/audio/coughing_1-58792-A-24.wav, len-of-chunks:1 +- id: e5ecf968737311ec83fd2fdc0fc83b8e score: 0.923869, cat input_audio: AudioCLIP/demo/audio/cat_3-95694-A-5.wav, len-of-chunks:1 +- id: e5ed00de737311ec83fd2fdc0fc83b8e score: 0.845475, cat input_audio: AudioCLIP/demo/audio/car_horn_1-24074-A-43.wav, len-of-chunks:1 +- id: e5ed0778737311ec83fd2fdc0fc83b8e score: 0.928283, cat input_audio: AudioCLIP/demo/audio/alarm_clock_3-120526-B-37.wav, len-of-chunks:1 +- id: e5ed0dcc737311ec83fd2fdc0fc83b8e score: 0.919222, alarm_clock

As above for 5 inputs only 2 outputs are correct. Please let me know why is that so ? Since AudioClipEncoder is giving correct outputs for all audios.

opened by snaaz21 55

[Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

Describe your proposal/problem

Dear Jina Team,

Recently, I tried nlp-simple example and replaced TransformerTorchEncoder with TextPaddlehubEncoder. However, I got the following error.

Traceback (most recent call last):
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 73, in _load_executor
    self._executor = BaseExecutor.load_config(
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 531, in load_config
    return JAML.load(tag_yml, substitute=False)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 89, in load
    r = yaml.load(stream, Loader=JinaLoader)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
    data = self.construct_object(node)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
    data = constructor(self, node)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 422, in _from_yaml
    return get_parser(cls, version=data.get('version', None)).parse(cls, data)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/parsers/executor/legacy.py", line 130, in parse
    obj = cls(
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 82, in __call__
    getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 174, in _post_init_wrapper
    self.post_init()
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/hub/encoders/nlp/TextPaddlehubEncoder/__init__.py", line 38, in post_init
    self.model = hub.Module(name=self.model_name)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 171, in __new__
    module = cls.init_with_name(
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 263, in init_with_name
    user_module_cls = manager.install(name=name, version=version, source=source, update=update, branch=branch)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 188, in install
    return self._install_from_name(name, version)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 263, in _install_from_name
    return self._install_from_url(item['url'])
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 256, in _install_from_url
    return self._install_from_archive(file)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 361, in _install_from_archive
    return self._install_from_directory(directory)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 345, in _install_from_directory
    hub_module_cls = HubModule.load(self._get_normalized_path(module_info.name))
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 219, in load
    paddle.set_device(place)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/device.py", line 166, in set_device
    framework._set_expected_place(place)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 317, in _set_expected_place
    _set_dygraph_tracer_expected_place(place)
  File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 311, in _set_dygraph_tracer_expected_place
    _dygraph_tracer_._expected_place = place
OSError: (External)  Cuda error(3), initialization error.
  [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:229)

It seems to be related with https://github.com/PaddlePaddle/Paddle/issues/25185.

However, I have no idea how to fix this. Could you help me to figure it out? Thank you very much!

Environment

paddlehub==2.0.0 paddlenlp==2.0.1 paddlepaddle-gpu==2.0.2.post90 jina==1.3.0

Screenshots

area/core type/bug

opened by ryh95 47

No documents are found with BinaryPbIndexer in 1.1.2

Versions 1.0.11: everything works as expected 1.1.2: BinaryPbIndexer doesn't find any documents

Detailed description He Jina team, when I upgrade to 1.1.2 and reindex my dataset, the BinaryPbIndexer no longer works. It doesn't raise any errors, but I just can't find any documents. Just as a remark, I use custom ids which always consist of 33 characters. Where there any changes for the BinaryPbIndexer, the KVIndexDriver or the KVSearchDriver?
area/core priority/critical type/bug status/done

opened by janandreschweiger 39
Permanently delete documents from storage
Hey Jina team!

As described here documents persist the deletion and are only marked as inaccessible.

My question: Is there a possibility to delete documents from storage without reindexing everything again? Deletion performance isn't an issue for us, but reindexing everything is very inconvenient in our use case.

Thanks! I really appreciate your kind support.

Sub-tasks

[x] Delete from NpIndexer. @ #2046

[x] Delete from BinaryPbIndexer. @ https://github.com/jina-ai/jina/pull/2102

area/core priority/important-soon type/feature-request status/done
opened by ace-kay-law-neo 39

File seems to be closed when trying to index

Yes @JoanFM, using the prefetch argument in the flow-config solved the original problem. I tried to pass the batch_size and the request_size like you, but I still get the same error. I have some additional logs, which might indicate what causes the problem. As I said, the first few batches are indexed successfully, but at some point I get this error:

        [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
        [email protected][I]:#sent: 3 #recv: 4 sent_size: 13.8 MB recv_size: 1.2 MB
      [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸⚐
        [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸⚐
   [email protected][S]:artifacts of this executor (wrap_idx) is persisted to /var/huforce/fs/workspace/wrap_idx-0/wrap_idx.bin
[email protected][S]:artifacts of this executor (chunk_idx) is persisted to /var/huforce/fs/workspace/chunk_idx-0/chunk_idx.bin
      [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
      [email protected][I]:#sent: 3 #recv: 4 sent_size: 1.9 MB recv_size: 15.6 MB
        [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸doc_idx/ZEDRuntime▸⚐
DEBUG | Message: Successfully indexed 4 documents!
        [email protected][S]:artifacts of this executor (doc_idx_file) is persisted to /var/huforce/fs/workspace/doc_idx_file-0/doc_idx_file.bin
        [email protected][I]:#sent: 3 #recv: 4 sent_size: 6.3 KB recv_size: 2.2 MB
       | 📃     16 ⏱️ 262.9s 🐎 0.1/s      4   requests        [email protected][I]:send: 5 recv: 4 pending: 1
        [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸⚐
DEBUG | Message: File-Info: research_admission.txt
DEBUG | Message: File-Info: cloudcomputing.pptx
DEBUG | Message: File-Info: WS_2012_White_Paper_Hyper-V.pdf
DEBUG | Message: File-Info: writing-sql-queries.docx
        [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
        [email protected][I]:#sent: 5 #recv: 6 sent_size: 1.2 MB recv_size: 5.2 KB
        [email protected][I]:#sent: 4 #recv: 5 sent_size: 15.6 MB recv_size: 1.2 MB
      [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸⚐
        [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸⚐
      [email protected][I]:#sent: 4 #recv: 5 sent_size: 2.2 MB recv_size: 16.1 MB
DEBUG | Message: Encoding 552 paragraphs
      [email protected][E]:ValueError('write to closed file')
 add "--quiet-error" to suppress the exception details
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 198, in _msg_callback
    self._zmqlet.send_message(self._callback(msg))
  File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 184, in _callback
    self._pre_hook(msg)._handle(msg)._post_hook(msg)
  File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 177, in _handle
    self._executor(self.request_type)
  File "/usr/local/lib/python3.8/site-packages/jina/executors/__init__.py", line 595, in __call__
    d()
  File "/usr/local/lib/python3.8/site-packages/jina/drivers/__init__.py", line 381, in __call__
    self._apply_all(documents, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/jina/drivers/index.py", line 58, in _apply_all
    self.exec_fn(keys, values)
  File "/usr/local/lib/python3.8/site-packages/jina/executors/decorators.py", line 44, in arg_wrapper
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/jina/executors/indexers/keyvalue.py", line 178, in add
    self.write_handler.header.write(
ValueError: write to closed file
        [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime✖▸doc_idx/ZEDRuntime▸⚐
dev_ai            | DEBUG | Message: Successfully indexed 4 documents!

What's also strange is, that the failed documents for the chunk-idx are passed on to the doc-idx, as you can see in the logs.

My flow for indexing:

!AsyncFlow
with:
  logserver: false
  show-exc-info: true
  prefetch: 2
  # request-size: 2
  # batch_size: 2
pods:
  crafter:
    uses: pods/craft.yml
  encoder:
    uses: pods/encode.yml
    timeout_ready: 600000
  chunk_idx:
    uses: pods/chunk.yml
  doc_idx:
    uses: pods/doc.yml

Originally posted by @nero-nazok in https://github.com/jina-ai/jina/issues/2298#issuecomment-817084516

area/core type/bug

opened by JoanFM 32

DOCQA..

I am trying to parse a pdf & apply the DOCQA model to it.Roberta as well as BERT..!! But I am not getting right answer for it.. whereas independently it's works accurately... !!

This is my flow.yml

jtype: Flow
version: '1'
with:
  workspace: $JINA_WORKSPACE
  port_expose: $JINA_PORT
executors:
  - name: transformer
    uses: 'jinahub://TransformerTorchEncoder/v0.1'
    uses_with:
      device: 'cuda'
  - name: indexer
    uses: 'jinahub://SimpleIndexer/old'

This is my query.yml

jtype: Flow
version: '1'
with:
  workspace: $JINA_WORKSPACE
  port_expose: $JINA_PORT
executors:
  - name: transformer
    uses: 'jinahub://TransformerTorchEncoder/v0.1'
    uses_with:
      device: 'cuda'
  - name: indexer
    uses: 'jinahub://SimpleIndexer/old'
  - name: generator
    uses: Generator
    py_modules: "flows/generator_roberta.py"

opened by jyotikhetan 29

Gateway error: received an empty stream from the client
Description

I encountered an error in flow.post method while trying to run python app.py -t index in multires-lyrics-search after cloning this repository: examples. The error message was: [email protected][E]:receive an empty stream from the client! please check your client's inputs, you can use "Client.check_input(inputs)".

Environment

jina 2.0.18

jina-proto 0.0.85

jina-vcs-tag (unset)

libzmq 4.3.4

pyzmq 1.21.5

protobuf 3.20.0

proto-backend cpp

grpcio 1.44.0

pyyaml 6.0

python 3.7.13

platform Linux

platform-release 5.10.105-1-MANJARO

platform-version #1 SMP PREEMPT Fri Mar 11 14:12:33 UTC 2022

architecture x86_64

processor

uid 5025763201072

session-id 7d472a12-b722-11ec-a806-049226d49030

uptime 2022-04-08T15:58:49.863145

ci-vendor (unset)

JINA_ARRAY_QUANT (unset)

JINA_CONTROL_PORT (unset)

JINA_DEFAULT_HOST (unset)

JINA_DISABLE_UVLOOP (unset)

JINA_FULL_CLI (unset)

JINA_HUBBLE_REGISTRY (unset)

JINA_HUB_CACHE_DIR (unset)

JINA_HUB_ROOT (unset)

JINA_LOG_CONFIG (unset)

JINA_LOG_ID (unset)

JINA_LOG_LEVEL (unset)

JINA_LOG_NO_COLOR (unset)

JINA_LOG_WORKSPACE (unset)

JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)

JINA_POD_NAME (unset)

JINA_RANDOM_PORT_MAX (unset)

JINA_RANDOM_PORT_MIN (unset)

JINA_VCS_VERSION (unset)

JINA_MP_START_METHOD (unset)
opened by Jackal1586 28
feat: remove head for non sharded deployments
Remove heads for non sharded Deployments

Description

This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers. Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

This is a breaking change :boom: It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

[x] Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD

[x] Set all worker Pods in the Gateway (Outside K8s)

[x] Make the gateway send to worker Pods

[x] Dont create heads for Deployments with no shards

[x] Implement reducing after multiple needs in the Gateway

[x] Adapt tests

[x] Change docs

Closes #4503 (issue)
area/core area/helper area/testing size/XL area/docs area/cli area/daemon
opened by jacobowitz 27
Unable to run Jina on Raspberry Pi 4
I installed Jina using the installation instructions in the docs and I was unable to run even the Jina command without the error

free(): invalid pointer Aborted

I have tried installing Jina from master branch, 1.2.4, and from pip. Yet, the problem persists

Environment Can't actually run this command.

Latest version of Raspbian OS (aarm64), on a RPi4. Python 3.7.3

Screenshots

type/bug
opened by navanchauhan 25
Server error popping up while running jina backend
While running the backend file of Jina this file Link

TimeoutError('no response from sock tcp://0.0.0.0:60797 after timeout=5000ms, please check the following: is the server still online? is the network broken? are "port" correct? ') during <bound method BasePea.close of <jina.peapods.peas.BasePea object at 0x7f052ce70430>> add "--quiet-error" to suppress the exception details

Environment

jina 2.2.0

jina-proto 0.0.86

jina-vcs-tag (unset)

libzmq 4.3.4

pyzmq 1.19.5

protobuf 3.19.3

proto-backend cpp

grpcio 1.43.0

pyyaml 5.3.1

python 3.8.10

platform Linux

platform-release 4.4.0-19041-Microsoft

platform-version #1237-Microsoft Sat Sep 11 14:32:00 PST 2021

architecture x86_64

processor x86_64

uid 4459196994945

session-id 389ff0fc-7ba8-11ec-83b5-040e3cda5581

uptime 2022-01-22T22:54:57.138560

ci-vendor (unset)

JINA_ARRAY_QUANT (unset)

JINA_CONTROL_PORT (unset)

JINA_DEFAULT_HOST (unset)

JINA_DEFAULT_TIMEOUT_CTRL (unset)

JINA_DISABLE_UVLOOP (unset)

JINA_FULL_CLI (unset)

JINA_HUBBLE_REGISTRY (unset)

JINA_HUB_CACHE_DIR (unset)

JINA_HUB_ROOT (unset)

JINA_K8S_USE_TEST_PIP (unset)

JINA_LOG_CONFIG (unset)

JINA_LOG_ID (unset)

JINA_LOG_LEVEL (unset)

JINA_LOG_NO_COLOR (unset)

JINA_LOG_WORKSPACE (unset)

JINA_MP_START_METHOD (unset)

JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)

JINA_POD_NAME (unset)

JINA_RANDOM_PORT_MAX (unset)

JINA_RANDOM_PORT_MIN (unset)

JINA_VCS_VERSION (unset) Screenshots

area/community
opened by Vidit-Ostwal 24

`As-a-Service in 10 Extra Lines ` : aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe

Following the indications here: https://github.com/jina-ai/jina#as-a-service-in-10-extra-lines-

I'm trying to execute the following script:

# https://github.com/jina-ai/jina#as-a-service-in-10-extra-lines-

from jina import Document, DocumentArray, Executor, Flow, requests

# pre-processing step
class PreprocImg(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        for d in docs:
            (d.load_uri_to_image_blob()  # load
             .set_image_blob_normalization()  # normalize color
             .set_image_blob_channel_axis(-1, 0))  # switch color axis


# embedding step
class EmbedImg(Executor):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        import torchvision
        self.model = torchvision.models.resnet50(pretrained=True)        

    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        docs.embed(self.model)

# matching step
class MatchImg(Executor):
    _da = DocumentArray()

    @requests(on='/index')
    def index(self, docs: DocumentArray, **kwargs):
        self._da.extend(docs)

    @requests(on='/search')
    def foo(self, docs: DocumentArray, **kwargs):
        docs.match(self._da)
        for d in docs.traverse_flat('r,m'):  # only require for visualization
            d.convert_uri_to_datauri()  # convert to datauri
            d.pop('embedding', 'blob')  # remove unnecessary fields for save bandwidth

# Connect all Executors in a Flow, scale embedding to 3
f = Flow(port_expose=12345, protocol='http').add(uses=PreprocImg).add(uses=EmbedImg, replicas=3).add(uses=MatchImg)

# Index image data and serve REST query publicly
with f:
    f.post('/index', DocumentArray.from_files('img/*.jpg'), show_progress=True, request_size=8)
    f.block()

But I get these errors:

(vespa) [email protected]:~/vespa-jina/jina-sample-apps$ python3 AsAService-ResNet501ImageSearch.py
           [email protected][I]:🎉 Flow is ready to use!                                                  
    🔗 Protocol: 		HTTP
    🏠 Local access:	0.0.0.0:12345
    🔒 Private network:	192.168.1.7:12345
    🌐 Public address:	109.116.161.240:12345
    💬 Swagger UI:		http://localhost:12345/docs
    📚 Redoc:		http://localhost:12345/redoc
UserWarning: ignored unknown argument: ['12345']. (raised from /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/helper.py:691)
⠋ Working... ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:03 estimating... UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:189.) (raised from /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/docarray/ndarray/__init__.py:359)
⠼      ERROR ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:05:00 0% ETA: 1 day and 7 hours and 15 minutes and 13 seconds Task exception was never retrieved
future: <Task finished name='Task-302' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ClientOSError(32, 'Broken pipe')>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe
Task exception was never retrieved
future: <Task finished name='Task-78' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-169' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-166' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-163' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-75' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-177' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-72' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-157' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-154' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-69' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-150' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-147' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-66' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-144' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-141' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-138' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-136' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-180' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-172' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-175' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-81' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-84' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-160' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-133' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-130' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-127' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-124' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-121' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-118' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-115' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-112' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-109' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-106' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-103' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-99' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-96' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-206' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-204' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-93' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-201' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-199' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-87' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-90' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-192' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-189' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-183' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-186' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError
Task exception was never retrieved
future: <Task finished name='Task-300' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ServerDisconnectedError('Server disconnected')>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
Task exception was never retrieved
future: <Task finished name='Task-304' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ServerDisconnectedError('Server disconnected')>
Traceback (most recent call last):
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
    return await self.session.post(url=self.url, json=req_dict).__aenter__()
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected

OS: Ubuntu 20.04 python version : Python 3.8.12

How to solve the problem?

priority/critical

opened by raphael10-collab 22

feat(websocket): http healthcheck endpoint
Goal

Even though AWS ALB supports Websocket traffic, it can't do health checks for WebSocket. It can do health checks with HTTP protocol though - I'm enabling the same to add WebSocket gateway support in JCloud. Now, it throws bunch of 404s since the endpoint actually doesn't exist.

This PR adds an HTTP endpoint GET / for WebsocketGateway like for HTTPGateway (since both are FastAPI apps on Uvicorn server). I'm not sure whether this would help other load balancers, though.

[ ] check and update documentation. See guide and ask the team.

area/core size/S area/testing
opened by deepankarm 2
Rewrite k8s docs
[ ] mention image build process and link to corresponding doc section

[ ] config.yml is hard coded in the k8s executor yaml, warn about this

[ ] Specify that our reliance on service meshes is a k8s+gRPC "problem", not one that we introduced
opened by JohannesMessner 0
Docs: explain error modelling and failures
Explain how (connection) errors are handled, locally, in docker and k8s, and how errors are reported to the user. Also explain that retrying in service mesh is potentially dangerous because we also retry.

ToDo:

[ ] error retry mechanism

[ ] in-house solution

[ ] k8s and service mesh
opened by JohannesMessner 0

Releases(v3.4.9)

v3.4.9(May 26, 2022)
Release Note (3.4.9)

Release time: 2022-05-26 11:40:41

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🐞 Bug fixes

[674e8121] - success box ui (Han Xiao)

🍹 Other Improvements

[a2b5e304] - Revert "ci: add create-release.yml (#4759)" (#4835) (Han Xiao)

[f053d764] - version: the next version will be 3.4.9 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.6(May 23, 2022)
Release Note (3.4.6)

Release time: 2022-05-23 19:21:36

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🐞 Bug fixes

[049666d9] - code review (#4821) (Han Xiao)

[688370db] - dockerfile: install security updates in docker image (Han Xiao)

🍹 Other Improvements

[174dbde1] - add security analysis (Han Xiao)

[93db296a] - version: the next version will be 3.4.6 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.5(May 23, 2022)
Release Note (3.4.5)

Release time: 2022-05-23 12:00:14

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, Johannes Messner, ZiniuYu, Joan Fontanals, Zhaofeng Miao, Jina Dev Bot, 🙇

🆕 New Features

[9c1fda96] - executor: add self logger to executor (#4818) (Han Xiao)

[e553d913] - smart replica retry (#4794) (Johannes Messner)

[a9ed27ed] - add additional metrics (#4789) (samsja)

[c88d2cf5] - hubble: use fixed domain for hubble api (#4804) (Zhaofeng Miao)

🐞 Bug fixes

[aa2d7c8d] - dockerfile: install security updates in docker image (Han Xiao)

[87de3c09] - define metrics in worker before grpc server (#4816) (samsja)

[a2bc2302] - avoid closing loop at fork time (#4803) (Joan Fontanals)

[3fb3b419] - warn user when calling load config from flow instance (#4787) (samsja)

🧼 Code Refactoring

[93f06fc5] - simplify event loop usage (#4811) (Joan Fontanals)

📗 Documentation

[3a7d95b6] - add specification of executor and flow yaml (#4795) (Johannes Messner)

[10ab59f1] - fix Flow docs typo (#4815) (ZiniuYu)

[15d83ba8] - round robin gpu scheduling (#4805) (samsja)

🍹 Other Improvements

[94153117] - docs: update TOC (Jina Dev Bot)

[3c7d0b3e] - version: the next version will be 3.4.5 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.4(May 18, 2022)
Release Note (3.4.4)

Release time: 2022-05-18 12:54:06

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Delgermurun, Sha Zhou, Jina Dev Bot, Roshan Jossy, 🙇

🐞 Bug fixes

[0372f10f] - unsetenv not existing on Windows (#4800) (Joan Fontanals)

[99ccff1f] - remove executor secret from flow mermaid chart (#4801) (Delgermurun)

[d66e98c1] - qabot: hide the slot elements on the docs page (#4799) (Sha Zhou)

📗 Documentation

[f0f5948e] - tracking: remove adding utm_source to links (#4798) (Roshan Jossy)

🍹 Other Improvements

[32d85a50] - docs: update TOC (Jina Dev Bot)

[2b5efac1] - version: the next version will be 3.4.4 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.3(May 17, 2022)
Release Note (3.4.3)

Release time: 2022-05-17 10:52:53

🙇 We'd like to thank all contributors for this new release! In particular, samsja, Johannes Messner, Jina Dev Bot, Han Xiao, 🙇

🐞 Bug fixes

[a32bfe21] - pop the init time instead of normal access (#4791) (samsja)

[c663e291] - add check that jaml parses to correct class (#4790) (Johannes Messner)

🍹 Other Improvements

[ec923d02] - docs: update TOC (Jina Dev Bot)

[95c1519d] - fix readme (Han Xiao)

[6194244f] - version: the next version will be 3.4.3 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.2(May 13, 2022)
Release Note (3.4.2)

Release time: 2022-05-13 20:09:02

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🆕 New Features

[1dd09184] - deployment: add roundrobin cuda device assignment (#4786) (Han Xiao)

🍹 Other Improvements

[8c4d1761] - remove unnecessary steps on chore commit (Han Xiao)

[01503bcd] - version: the next version will be 3.4.2 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.1(May 13, 2022)
Release Note (3.4.1)

Release time: 2022-05-13 11:55:54

🙇 We'd like to thank all contributors for this new release! In particular, Johannes Messner, Delgermurun, Han Xiao, Roshan Jossy, Florian Hönicke, Jina Dev Bot, samsja, 🙇

🆕 New Features

[072a47a4] - better error messages when gateway can't connect to other deployment (#4677) (Johannes Messner)

[6514ca95] - hubble: executor push/pull user integration (#4770) (Delgermurun)

🐞 Bug fixes

[fb529399] - sandbox: send secret in the request (#4781) (Delgermurun)

[2e9484ef] - yaml jtype flow (#4776) (samsja)

🧼 Code Refactoring

[934ecd8f] - flow: improve success box (#4782) (Han Xiao)

[b0df40ae] - logging no print (#4777) (Florian Hönicke)

📗 Documentation

[ad5ed0dc] - tracking: fix utm source (#4780) (Roshan Jossy)

[7de44506] - add jina now logo (#4775) (Florian Hönicke)

🏁 Unit Test and CICD

[d7c48d5f] - hubble: fix failing test (#4783) (Delgermurun)

🍹 Other Improvements

[ec136dd8] - update announcement in readme and docs (Jina Dev Bot)

[13a954be] - docs: update TOC (Jina Dev Bot)

[36ca89d3] - version: the next version will be 3.4.1 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.4.0(May 10, 2022)
Highlights :star2:

Monitoring :computer:

Now Jina enables you to monitor the performance of your Executors and the complete Flow with the help of Prometheus and allows you to build a Grafana dashboard to gain better insights on how your system works.

from jina import Flow f = Flow(monitoring=True).add().add() with f: f.block()

#4636 #4635 #4638 #4672 #4689 #4687 #4735 #4738 #4737 #4758 #4752 #4768

Robustness :muscle:

We have improved the robustness of the system and the network communication between microservices in a Flow and have put a lot of work on getting better information of potential failures in the system.

Changes include:

Adding a timeout parameter for the Gateway for its grpc communication with Executors #4660

Ensuring a graceful shutdown of Runtimes in Kubernetes to avoid requests loss #4682

Adding an extensive test for handling Pod failues in Kubernetes #4743

Better error reporting of errors happening in the Head of a sharded Executor #4640

Better error reporting when Flow fails to start #4652

UI improvements :haircut:

Jina comes with a better UI and a better logging system to help the user get better information from the logs in the best format based on rich.

#4614 #4697 #4724 #4736 #4739

Other changes

Remove unnecessary argument from Executors runtime_backend (#4644)

Pass environment variables to Gateway and Head Runtimes (#4646)

Add resource layer with gpu in docker-compose files when to_docker_compose is called for Services requiring it (#4718)

You can have inmutable tags when pushing Executors into the Hub (#4676)

target_executor parameter is considered a regex pattern and not a exact string. All Executors matching this regex will receive the request. (#4756)

Gateway only sends request to Executor if the endpoint is exposed by it (#4756)

Bug fixes :bug:

Avoid leaking file descriptors when client sends requests (#4734)

Fix flow plotting when private Executor is used (#4643)

Executor could not start processes internally (#4692)

Deadlock on Executor start due to ModuleLock error (#4693)

Flow early stop when startup failure (#4726)

Breaking changes :warning:

target_executor is treated as a regex pattern and not as a string.

Before:

from docarray import DocumentArray from jina import Flow, Executor, requests class Exec(Executor): @requests def foo(self, docs, **kwargs): for doc in docs: doc.text += 'calling foo ' f = Flow().add(name='exec', uses=Exec).add(name='exec_after', uses=Exec) with f: res = f.post(on='/', inputs=DocumentArray.empty(1), target_executor='exec') print(res.texts)

Before, it would give:

['calling foo ']

Now:

['calling foo calling foo ']

:heart: Thanks to the community for all the support and hard work!
Source code(tar.gz)
Source code(zip)
v3.3.25(May 5, 2022)
Release Note (3.3.25)

Release time: 2022-05-05 17:39:33

🙇 We'd like to thank all contributors for this new release! In particular, Tobias Jacobowitz, Roshan Jossy, Jina Dev Bot, 🙇

🐞 Bug fixes

[c1f0ae51] - close loop from run_async (#4734) (Tobias Jacobowitz)

📗 Documentation

[36ce8359] - tracking: update refer links to other docs with utm_source (#4744) (Roshan Jossy)

🍹 Other Improvements

[913d1f31] - docs: update TOC (Jina Dev Bot)

[d5c9967c] - version: the next version will be 3.3.25 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.24(May 4, 2022)
Release Note (3.3.24)

Release time: 2022-05-04 14:52:39

🙇 We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, samsja, Jina Dev Bot, 🙇

🆕 New Features

[f9f86004] - inherit monitoring arg from Flow (#4738) (samsja)

🍹 Other Improvements

[58847187] - grpc version>=1.46.0 (#4746) (Deepankar Mahapatro)

[e3c45bf5] - docs: update TOC (Jina Dev Bot)

[c4175f95] - version: the next version will be 3.3.24 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.23(May 2, 2022)
Release Note (3.3.23)

Release time: 2022-05-02 20:10:11

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, Jina Dev Bot, 🙇

🐞 Bug fixes

[de381ba0] - logger: make log format more compact (#4739) (Han Xiao)

[944afbc5] - move prometheus depedency to perf (#4735) (samsja)

[3a1a8f64] - rich console logs (#4724) (samsja)

🍹 Other Improvements

[4d1e4685] - fix rich log format string (#4736) (Han Xiao)

[1ddc3953] - fix rich log format string (Han Xiao)

[f8b994dc] - docs: update TOC (Jina Dev Bot)

[2f08c1b6] - version: the next version will be 3.3.23 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.22(Apr 30, 2022)
Release Note (3.3.22)

Release time: 2022-04-30 18:43:18

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🐞 Bug fixes

[1516c276] - flow: early stop (#4726) (Han Xiao)

🍹 Other Improvements

[930d4bab] - remove redundant colored (Han Xiao)

[5152c983] - docs: update TOC (Jina Dev Bot)

[bca843ad] - version: the next version will be 3.3.22 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.21(Apr 29, 2022)
Release Note (3.3.21)

Release time: 2022-04-29 19:17:10

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🍹 Other Improvements

[2b2b2e61] - remove redundant code (Han Xiao)

[5ad319ef] - version: the next version will be 3.3.21 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.20(Apr 29, 2022)
Release Note (3.3.20)

Release time: 2022-04-29 13:48:42

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Jina Dev Bot, 🙇

🆕 New Features

[55d1f44e] - allow jina now to unset jina_check_version (#4722) (Joan Fontanals)

[615e3efe] - monitoring support for k8s (#4687) (samsja)

🐞 Bug fixes

[976792e9] - edit conda recipe generator (#4723) (Joan Fontanals)

🍹 Other Improvements

[f5dacb1c] - update announcement in readme and docs (Jina Dev Bot)

[16b6f154] - docs: update TOC (Jina Dev Bot)

[2ff693f3] - version: the next version will be 3.3.20 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.19(Apr 28, 2022)
Release Note (3.3.19)

Release time: 2022-04-28 16:14:00

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Yanlong Wang, Jina Dev Bot, 🙇

🆕 New Features

[93bda042] - allow user not to print version check print (#4720) (Joan Fontanals)

[19e5cd7e] - immutable tags for executors (#4676) (Yanlong Wang)

🐞 Bug fixes

[ac204dcb] - rename immutable-tag to protected-tag (#4721) (Yanlong Wang)

🍹 Other Improvements

[a884f616] - version: the next version will be 3.3.19 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.18(Apr 28, 2022)
Release Note (3.3.18)

Release time: 2022-04-28 13:17:43

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Johannes Messner, Jina Dev Bot, 🙇

🐞 Bug fixes

[0def13df] - add resource layer in docker compose gpu (#4718) (Joan Fontanals)

[e57f9524] - remove version check from cli (#4716) (Joan Fontanals)

[07ae316d] - fix grpcio-reflection version (#4715) (Joan Fontanals)

[917604de] - update failing test (#4711) (samsja)

[59340e05] - the cd (#4714) (Johannes Messner)

🍹 Other Improvements

[e75efaef] - docs: update TOC (Jina Dev Bot)

[87b4b035] - version: the next version will be 3.3.18 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.17(Apr 27, 2022)
Release Note (3.3.17)

Release time: 2022-04-27 21:10:33

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🍹 Other Improvements

[9f83ab6b] - readme badge (Han Xiao)

[b2e11974] - sort and optimize imports (#4713) (Han Xiao)

[22d5ce67] - update readme (Han Xiao)

[d2423b27] - version: the next version will be 3.3.17 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.16(Apr 27, 2022)
Release Note (3.3.16)

Release time: 2022-04-27 19:38:37

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

[4bafca88] - logger: use docker log handler for container (#4712) (Han Xiao)

[bb227615] - fix add deploy resources when gpus detected (#4706) (Joan Fontanals)

🍹 Other Improvements

[955adb40] - version: the next version will be 3.3.16 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.15(Apr 27, 2022)
Release Note (3.3.15)

Release time: 2022-04-27 15:42:23

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Han Xiao, Jina Dev Bot, 🙇

🧼 Code Refactoring

[a8f50dd0] - change way of importing in cli (#4709) (Joan Fontanals)

[734edbc2] - some UI improvement (#4705) (Han Xiao)

🍹 Other Improvements

[b1bff161] - version: the next version will be 3.3.15 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.14(Apr 27, 2022)
Release Note (3.3.14)

Release time: 2022-04-27 10:50:31

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

[9a579cd6] - use warnings and threading (#4703) (Joan Fontanals)

🍹 Other Improvements

[06fc848c] - version: the next version will be 3.3.14 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.13(Apr 27, 2022)
Release Note (3.3.13)

Release time: 2022-04-27 09:08:20

🙇 We'd like to thank all contributors for this new release! In particular, felix-wang, Jina Dev Bot, 🙇

🐞 Bug fixes

[e56d787c] - parsing requirements of executor (#4698) (felix-wang)

🍹 Other Improvements

[d1c44920] - version: the next version will be 3.3.13 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.12(Apr 27, 2022)
Release Note (3.3.12)

Release time: 2022-04-27 08:57:34

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Yanlong Wang, Jina Dev Bot, 🙇

🐞 Bug fixes

[a3b71c72] - parsers: clearify flow args (#4701) (Han Xiao)

[babd97d0] - hubio: improve error message from hubble (#4674) (Yanlong Wang)

🍹 Other Improvements

[524bedd5] - docs: update TOC (Jina Dev Bot)

[7649d5d9] - version: the next version will be 3.3.12 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.11(Apr 26, 2022)
Release Note (3.3.11)

Release time: 2022-04-26 14:47:52

🙇 We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, Tobias Jacobowitz, Jina Dev Bot, Joan Fontanals, 🙇

🆕 New Features

[25b8df17] - graceful shutdown (#4682) (Tobias Jacobowitz)

🐞 Bug fixes

[e23ff11a] - pod does not start runtime process as daemon (#4692) (Joan Fontanals)

🍹 Other Improvements

[05222069] - disable console color with envvar (#4697) (Deepankar Mahapatro)

[655213d0] - docs: update TOC (Jina Dev Bot)

[d9b5b718] - version: the next version will be 3.3.11 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.10(Apr 25, 2022)
Release Note (3.3.10)

Release time: 2022-04-25 21:00:48

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

[5af4efe3] - fix deadlock on import (#4693) (Joan Fontanals)

🍹 Other Improvements

[a8329d66] - version: the next version will be 3.3.10 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.9(Apr 25, 2022)
Release Note (3.3.9)

Release time: 2022-04-25 19:59:19

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

[162a59b6] - fix target executor in http (#4690) (Joan Fontanals)

🍹 Other Improvements

[278a6992] - version: the next version will be 3.3.9 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.8(Apr 25, 2022)
Release Note (3.3.8)

Release time: 2022-04-25 17:42:17

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

[81a0b8d5] - fix configuration of addresses in docker compose (#4684) (Joan Fontanals)

🍹 Other Improvements

[e9a4d79c] - version: the next version will be 3.3.8 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.7(Apr 25, 2022)
Release Note (3.3.7)

Release time: 2022-04-25 14:16:33

🙇 We'd like to thank all contributors for this new release! In particular, samsja, Tobias Jacobowitz, Johannes Messner, joschkabraun, Joan Fontanals, Jina Dev Bot, cristian, felix-wang, 🙇

🆕 New Features

[6ac879e4] - add linkerd details in contributing (#4686) (samsja)

[59b76269] - add timeout_send (#4660) (Tobias Jacobowitz)

[4c4fc9e2] - monitoring: count the number of processed documents (#4672) (samsja)

[b3b139be] - pass environment to head and gateway (#4646) (Joan Fontanals)

🐞 Bug fixes

[9b8deaa2] - dependency: prometheus client should be a core dependency (#4689) (samsja)

[96a83469] - better error messages when flow fails to start (#4652) (Johannes Messner)

[4168d44d] - set the multiprocessing start_method spawn (#4650) (felix-wang)

📗 Documentation

[85b597ab] - fixed typos in limiting outstanding requests (#4611) (joschkabraun)

[b6aaa3ef] - add docs guide (#4659) (cristian)

[0c1468a7] - normalize usage of admonition boxes (#4662) (Johannes Messner)

🍹 Other Improvements

[b086f0d9] - docs: update TOC (Jina Dev Bot)

[e7bd24ab] - version: the next version will be 3.3.7 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.6(Apr 22, 2022)
Release Note (3.3.6)

Release time: 2022-04-22 07:01:51

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Jina Dev Bot, Johannes Messner, 🙇

🐞 Bug fixes

[955a08d8] - fix circular import (#4649) (Joan Fontanals)

[af81dfe9] - reorder correctly the remark page (#4665) (samsja)

📗 Documentation

[4a0a878e] - make all how-to titles sentence case (#4663) (Johannes Messner)

🍹 Other Improvements

[0c8d6371] - docs: update TOC (Jina Dev Bot)

[d1add5a6] - version: the next version will be 3.3.6 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.5(Apr 21, 2022)
Release Note (3.3.5)

Release time: 2022-04-21 14:49:03

🙇 We'd like to thank all contributors for this new release! In particular, Tobias Jacobowitz, Jina Dev Bot, 🙇

🐞 Bug fixes

[8c53323e] - sandbox usage from k8s/docker-compose (#4671) (Tobias Jacobowitz)

🍹 Other Improvements

[bf4c2be2] - version: the next version will be 3.3.5 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)
v3.3.4(Apr 21, 2022)
Release Note (3.3.4)

Release time: 2022-04-21 12:56:57

🙇 We'd like to thank all contributors for this new release! In particular, cristian, Joan Fontanals, felix-wang, Tobias Jacobowitz, Johannes Messner, Jina Dev Bot, 🙇

🐞 Bug fixes

[2c72e770] - pass extra search paths given from args (#4651) (felix-wang)

[8a3083f8] - relax time constraint in block exec test (#4653) (Tobias Jacobowitz)

[e92e26cc] - better reporting of errors in the head (#4640) (Johannes Messner)

🧼 Code Refactoring

[b76d4774] - remove runtime backend cli arg (#4644) (Joan Fontanals)

📗 Documentation

[d2161e66] - fix admonition in access flow page (#4655) (cristian)

🍹 Other Improvements

[9140b134] - version: the next version will be 3.3.4 (Jina Dev Bot)

Source code(tar.gz)
Source code(zip)

An easier way to build neural search on the cloud

Related tags

Overview

Installation

Jina "Hello, World!" 👋 🌍

Covid-19 Chatbot

Get Started

🥚 Fundamental

CRUD Functions

Document

MultimodalDocument

Fusion Embeddings from Different Modalities

Flow

🐣 Basic

Feed Data

Fetch Result

Add Logic

Inter & Intra Parallelism

Decentralized Flow

Asynchronous Flow

🐥 Breakdown of hello-world

Customize Encoder

Test Encoder in Flow

Parallelism & Batching

Add Data Indexer

Compose Flow from YAML

Search

Evaluation

REST Interface

Learn

Examples (View all)

📄

📄

📄

📄

🖼️

🎧

🎞️

Documentation

Contributing

Contributors ✨

Community

Open Governance

Join Us

License

Issues

Remove heads for non sharded Deployments

Description

Releases(v3.4.9)

v3.4.9(May 26, 2022)

Release Note (3.4.9)

🐞 Bug fixes

🍹 Other Improvements

v3.4.6(May 23, 2022)

Release Note (3.4.6)

🐞 Bug fixes

🍹 Other Improvements

v3.4.5(May 23, 2022)

Release Note (3.4.5)

🆕 New Features

🐞 Bug fixes

🧼 Code Refactoring

📗 Documentation

🍹 Other Improvements

v3.4.4(May 18, 2022)

Release Note (3.4.4)

🐞 Bug fixes

📗 Documentation

🍹 Other Improvements

v3.4.3(May 17, 2022)

Release Note (3.4.3)

🐞 Bug fixes

🍹 Other Improvements

v3.4.2(May 13, 2022)

Release Note (3.4.2)

🆕 New Features

🍹 Other Improvements

v3.4.1(May 13, 2022)

Release Note (3.4.1)

🆕 New Features

🐥 Breakdown of `hello-world`

Release Note (`3.4.9`)

Release Note (`3.4.6`)

Release Note (`3.4.5`)

Release Note (`3.4.4`)

Release Note (`3.4.3`)

Release Note (`3.4.2`)

Release Note (`3.4.1`)

Release Note (`3.3.25`)

Release Note (`3.3.24`)

Release Note (`3.3.23`)

Release Note (`3.3.22`)

Release Note (`3.3.21`)

Release Note (`3.3.20`)

Release Note (`3.3.19`)

Release Note (`3.3.18`)