An easier way to build neural search on the cloud

Overview

Jina banner

An easier way to build neural search on the cloud

Jina Python 3.7 3.8 3.9 PyPI Docker Image Version (latest semver) CI CD codecov

Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g. text, images, video, audio) on the cloud.

⏱️ Time Saver - The design pattern of neural search systems, from zero to a production-ready system in minutes.

🍱 Full-Stack Ownership - Keep an end-to-end stack ownership of your solution, avoid the integration pitfalls with fragmented, multi-vendor, generic legacy tools.

🌌 Universal Search - Large-scale indexing and querying of unstructured data: video, image, long/short text, music, source code, etc.

🧠 First-Class AI Models - First-class support for state-of-the-art AI models, easily usable and extendable with a Pythonic interface.

🌩️ Fast & Cloud Ready - Decentralized architecture from day one. Scalabe & cloud-native by design: enjoy containerizing, distributing, sharding, async, REST/gRPC/WebSocket.

❤️ Made with Love - Never compromise on quality, actively maintained by a passionate full-time, venture-backed team.


DocsHello WorldQuick StartLearnExamplesContributeJobsWebsiteSlack

Installation

📦
x86/64,arm/v6,v7,v8 (Apple M1)
On Linux/macOS & Python 3.7/3.8/3.9 Docker Users
Standard pip install -U jina docker run jinaai/jina:latest
Daemon pip install -U "jina[daemon]" docker run --network=host jinaai/jina:latest-daemon
With Extras pip install -U "jina[devel]" docker run jinaai/jina:latest-devel
Dev/Pre-Release pip install --pre jina docker run jinaai/jina:master

Version identifiers are explained here. To install Jina with extra dependencies please refer to the docs. Jina can run on Windows Subsystem for Linux. We welcome the community to help us with native Windows support.

Jina "Hello, World!" 👋 🌍

Just starting out? Try Jina's "Hello, World" - a simple image neural search demo for Fashion-MNIST. No extra dependencies needed, simply run:

jina hello-world  # more options in --help

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html
# replace "open" with "xdg-open" on Linux
Click here to see console output

hello world console output

This downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and after running opens a webpage and shows results:

Covid-19 Chatbot

For NLP engineers, we provide a simple chatbot demo for answering Covid-19 questions. You will need PyTorch and Transformers, which can be installed along with Jina:

pip install "jina[torch,transformers]"
jina hello-world-chatbot

This downloads CovidQA dataset and tells Jina to index 418 question-answer pairs with DistilBERT. The index process takes about 1 minute on CPU. Then it opens a webpage where you can input questions and ask Jina.

Get Started

🥚 CRUD FunctionsDocumentFlow
🐣 Feed DataFetch ResultAdd LogicInter & Intra ParallelismDecentralizeAsynchronous
🐥 Customize EncoderTest EncoderParallelism & BatchingAdd Data IndexerCompose Flow from YAMLSearchEvaluationREST Interface

🥚 Fundamental

CRUD Functions

First we look at basic CRUD operations. In Jina, CRUD corresponds to four functions: index (create), search (read), update, and delete. With Documents below as an example:

import numpy as np
from jina import Document
docs = [Document(id='🐲', embedding=np.array([0, 0]), tags={'guardian': 'Azure Dragon', 'position': 'East'}),
        Document(id='🐦', embedding=np.array([1, 0]), tags={'guardian': 'Vermilion Bird', 'position': 'South'}),
        Document(id='🐢', embedding=np.array([0, 1]), tags={'guardian': 'Black Tortoise', 'position': 'North'}),
        Document(id='🐯', embedding=np.array([1, 1]), tags={'guardian': 'White Tiger', 'position': 'West'})]

Let's build a Flow with a simple indexer:

from jina import Flow
f = Flow().add(uses='_index')

Document and Flow are basic concepts in Jina, which will be explained later. _index is a built-in embedding + structured storage that one can use out of the box.

Index
# save four docs (both embedding and structured info) into storage
with f:
    f.index(docs, on_done=print)
Search
# retrieve top-3 neighbours of 🐲, this print 🐲🐦🐢 with score 0, 1, 1 respectively
with f:
    f.search(docs[0], top_k=3, on_done=lambda x: print(x.docs[0].matches))
{"id": "🐲", "tags": {"guardian": "Azure Dragon", "position": "East"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1}
{"id": "🐦", "tags": {"position": "South", "guardian": "Vermilion Bird"}, "embedding": {"dense": {"buffer": "AQAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1}
{"id": "🐢", "tags": {"guardian": "Black Tortoise", "position": "North"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAABAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "🐲"}, "adjacency": 1}
Update
# update 🐲 embedding in the storage
docs[0].embedding = np.array([1, 1])
with f:
    f.update(docs[0])
Delete
# remove 🐦🐲 Documents from the storage
with f:
    f.delete(['🐦', '🐲'])

Document

Document is Jina's primitive data type. It can contain text, image, array, embedding, URI, and accompanied by rich meta information. To construct a Document, one can use:

import numpy
from jina import Document

doc1 = Document(content=text_from_file, mime_type='text/x-python')  # a text document contains python code
doc2 = Document(content=numpy.random.random([10, 10]))  # a ndarray document

Document can be recurred both vertically and horizontally to have nested documents and matched documents. To better see the recursive structure of a document, one can use .plot() function. If you are using JupyterLab/Notebook, all Document objects will be auto-rendered.

import numpy
from jina import Document

d0 = Document(id='🐲', embedding=np.array([0, 0]))
d1 = Document(id='🐦', embedding=np.array([1, 0]))
d2 = Document(id='🐢', embedding=np.array([0, 1]))
d3 = Document(id='🐯', embedding=np.array([1, 1]))

d0.chunks.append(d1)
d0.chunks[0].chunks.append(d2)
d0.matches.append(d3)

d0.plot()  # simply `d0` on JupyterLab
Click here to see more about MultimodalDocument

MultimodalDocument

A MultimodalDocument is a document composed of multiple Document from different modalities (e.g. text, image, audio).

Jina provides multiple ways to build a multimodal Document. For example, one can provide the modality names and the content in a dict:

from jina import MultimodalDocument
document = MultimodalDocument(modality_content_map={
    'title': 'my holiday picture',
    'description': 'the family having fun on the beach',
    'image': PIL.Image.open('path/to/image.jpg')
})

One can also compose a MultimodalDocument from multiple Document directly:

from jina.types import Document, MultimodalDocument

doc_title = Document(content='my holiday picture', modality='title')
doc_desc = Document(content='the family having fun on the beach', modality='description')
doc_img = Document(content=PIL.Image.open('path/to/image.jpg'), modality='image')
doc_img.tags['date'] = '10/08/2019'

document = MultimodalDocument(chunks=[doc_title, doc_description, doc_img])
Fusion Embeddings from Different Modalities

To extract fusion embeddings from different modalities Jina provides BaseMultiModalEncoder abstract class, which has a unqiue encode interface.

def encode(self, *data: 'numpy.ndarray', **kwargs) -> 'numpy.ndarray':
    ...

MultimodalDriver provides data to the MultimodalDocument in the correct expected order. In this example below, image embedding is passed to the endoder as the first argument, and text as the second.

!MyMultimodalEncoder
with:
  positional_modality: ['image', 'text']
requests:
  on:
    [IndexRequest, SearchRequest]:
      - !MultiModalDriver {}

Interested readers can refer to jina-ai/example: how to build a multimodal search engine for image retrieval using TIRG (Composing Text and Image for Image Retrieval) for the usage of MultimodalDriver and BaseMultiModalEncoder in practice.

Flow

Jina provides a high-level Flow API to simplify building CRUD workflows. To create a new Flow:

from jina import Flow
f = Flow().add()

This creates a simple Flow with one Pod. You can chain multiple .add()s in a single Flow.

To visualize the Flow, simply chain it with .plot('my-flow.svg'). If you are using a Jupyter notebook, the Flow object will be displayed inline without plot.

Gateway is the entrypoint of the Flow.

Get the vibe? Now we are talking! Let's learn more about the basic concepts and features in Jina.


🥚 CRUD FunctionsDocumentFlow
🐣 Feed DataFetch ResultAdd LogicInter & Intra ParallelismDecentralizeAsynchronous
🐥 Customize EncoderTest EncoderParallelism & BatchingAdd Data IndexerCompose Flow from YAMLSearchEvaluationREST Interface

🐣 Basic

Feed Data

To use a Flow, open it via with context manager, like you would open a file in Python. Now let's create some empty document and index it:

from jina import Document

with Flow().add() as f:
    f.index((Document() for _ in range(10)))

Flow supports CRUD operations: index, search, update, delete. Besides, it also provides sugary syntax on ndarray, csv, ndjson and arbitrary files.

Input Example on index/search Explain
numpy.ndarray
with f:
  f.index_ndarray(numpy.random.random([4,2]))

Input four Document, each document.blob is a ndarray([2])

CSV
with f, open('index.csv') as fp:
  f.index_csv(fp1, field_resolver={'pic_url': 'uri'})

Each line in the index.csv is constructed as Document, CSV's field pic_url is mapped to document.uri.

JSON Lines/ndjson/LDJSON
with f, open('index.ndjson') as fp:
  f.index_ndjson(fp1, field_resolver={'question_id': 'id'})

Each line in index.ndjson is constructed as Document, JSON's field question_id is mapped to document.id.

Files with wildcard
with f:
  f.index_files(['/tmp/*.mp4', '/tmp/*.pdf'])

Each file captured is constructed as a Document, whose content (text, blob, buffer) is auto-guessed & filled.

Fetch Result

Once a request is done, callback functions are fired. Jina Flow implements Promise-like interface, you can add callback functions on_done, on_error, on_always to hook different events. In the example below, our Flow passes the message then prints the result when successful. If something wrong, it beeps. Finally, the result is written to output.txt.

def beep(*args):
    # make a beep sound
    import os
    os.system('echo -n "\a";')

with Flow().add() as f, open('output.txt', 'w') as fp:
    f.index(numpy.random.random([4, 5, 2]),
            on_done=print, on_error=beep, on_always=lambda x: fp.write(x.json()))

Add Logic

To add logic to the Flow, use the uses parameter to attach a Pod with an Executor. uses accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.

f = (Flow().add(uses='MyBertEncoder')  # class name of a Jina Executor
           .add(uses='docker://jinahub/pod.encoder.dummy_mwu_encoder:0.0.6-0.9.3')  # the image name
           .add(uses='myencoder.yml')  # YAML serialization of a Jina Executor
           .add(uses='!WaveletTransformer | {freq: 20}')  # inline YAML config
           .add(uses='_pass')  # built-in shortcut executor
           .add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}}))  # dict config object with __cls keyword

The power of Jina lies in its decentralized architecture: each add creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.

Inter & Intra Parallelism

Chaining .add()s creates a sequential Flow. For parallelism, use the needs parameter:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', needs='gateway')
           .needs(['p1','p2', 'p3'], name='r1').plot())

p1, p2, p3 now subscribe to Gateway and conduct their work in parallel. The last .needs() blocks all Pods until they finish their work. Note: parallelism can also be performed inside a Pod using parallel:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', parallel=3)
           .needs(['p1','p3'], name='r1').plot())

Decentralized Flow

A Flow does not have to be local-only, one can put any Pod to remote(s). In the example below, with the host keyword gpu-pod is put to a remote machine for parallelization, whereas other pods stay local. Extra file dependencies that need to be uploaded are specified via the upload_files keyword.

123.456.78.9
# have docker installed
docker run --name=jinad --network=host -v /var/run/docker.sock:/var/run/docker.sock jinaai/jina:latest-daemon --port-expose 8000
# to stop it
docker rm -f jinad
Local
import numpy as np
from jina import Flow

f = (Flow()
     .add()
     .add(name='gpu_pod',
          uses='mwu_encoder.yml',
          host='123.456.78.9:8000',
          parallel=2,
          upload_files=['mwu_encoder.py'])
     .add())

with f:
    f.index_ndarray(np.random.random([10, 100]), output=print)

We provide a demo server on cloud.jina.ai:8000, give the following snippet a try!

from jina import Flow

with Flow().add().add(host='cloud.jina.ai:8000') as f:
    f.index(['hello', 'world'])

Asynchronous Flow

Synchronous from outside, Jina runs asynchronously underneath: it manages the eventloop(s) for scheduling the jobs. If the user wants more control over the eventloop, then AsyncFlow comes to use.

Unlike Flow, the CRUD of AsyncFlow accepts input & output functions as async generator. This is useful when your data sources involve other asynchronous libraries (e.g. motor for MongoDB):

from jina import AsyncFlow

async def input_fn():
    for _ in range(10):
        yield Document()
        await asyncio.sleep(0.1)

with AsyncFlow().add() as f:
    async for resp in f.index(input_fn):
        print(resp)

AsyncFlow is particular useful when Jina is using as part of the integration, where another heavy-lifting job is running concurrently:

async def run_async_flow_5s():  # WaitDriver pause 5s makes total roundtrip ~5s
    with AsyncFlow().add(uses='- !WaitDriver {}') as f:
        async for resp in f.index_ndarray(numpy.random.random([5, 4])):
            print(resp)

async def heavylifting():  # total roundtrip takes ~5s
    print('heavylifting other io-bound jobs, e.g. download, upload, file io')
    await asyncio.sleep(5)
    print('heavylifting done after 5s')

async def concurrent_main():  # about 5s; but some dispatch cost, can't be just 5s, usually at <7s
    await asyncio.gather(run_async_flow_5s(), heavylifting())

if __name__ == '__main__':
    asyncio.run(concurrent_main())

AsyncFlow is very useful when using Jina inside the Jupyter Notebook. As Jupyter/ipython already manages an eventloop and thanks to autoawait, AsyncFlow can run out-of-the-box in Jupyter.

That's all you need to know for understanding the magic behind hello-world. Now let's dive into it!


🥚 CRUD FunctionsDocumentFlow
🐣 Feed DataFetch ResultAdd LogicInter & Intra ParallelismDecentralizeAsynchronous
🐥 Customize EncoderTest EncoderParallelism & BatchingAdd Data IndexerCompose Flow from YAMLSearchEvaluationREST Interface

🐥 Breakdown of hello-world

Customize Encoder

Let's first build a naive image encoder that embeds images into vectors using an orthogonal projection. To do this, we simply inherit from BaseImageEncoder: a base class from the jina.executors.encoders module. We then override its __init__() and encode() methods.

import numpy as np
from jina.executors.encoders import BaseImageEncoder

class MyEncoder(BaseImageEncoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        np.random.seed(1337)
        H = np.random.rand(784, 64)
        u, s, vh = np.linalg.svd(H, full_matrices=False)
        self.oth_mat = u @ vh

    def encode(self, data: 'np.ndarray', *args, **kwargs):
        return (data.reshape([-1, 784]) / 255) @ self.oth_mat

Jina provides a family of Executor classes, which summarize frequently-used algorithmic components in neural search. This family consists of encoders, indexers, crafters, evaluators, and classifiers, each with a well-designed interface. You can find the list of all 107 built-in executors here. If they don't meet your needs, inheriting from one of them is the easiest way to bootstrap your own Executor. Simply use our Jina Hub CLI:

pip install jina[hub] && jina hub new

Test Encoder in Flow

Let's test our encoder in the Flow with some synthetic data:

def validate(req):
    assert len(req.docs) == 100
    assert NdArray(req.docs[0].embedding).value.shape == (64,)

f = Flow().add(uses='MyEncoder')

with f:
    f.index_ndarray(numpy.random.random([100, 28, 28]), on_done=validate)

All good! Now our validate function confirms that all one hundred 28x28 synthetic images have been embedded into 100x64 vectors.

Parallelism & Batching

By setting a larger input, you can play with batch_size and parallel:

f = Flow().add(uses='MyEncoder', parallel=10)

with f:
    f.index_ndarray(numpy.random.random([60000, 28, 28]), batch_size=1024)

Add Data Indexer

Now we need to add an indexer to store all the embeddings and the image for later retrieval. Jina provides a simple numpy-powered vector indexer NumpyIndexer, and a key-value indexer BinaryPbIndexer. We can combine them in a single YAML file:

!CompoundIndexer
components:
  - !NumpyIndexer
    with:
      index_filename: vec.gz
  - !BinaryPbIndexer
    with:
      index_filename: chunk.gz
metas:
  workspace: ./
  • ! tags a structure with a class name
  • with defines arguments for initializing this class object.

Essentially, the above YAML config is equivalent to the following Python code:

from jina.executors.indexers.vector import NumpyIndexer
from jina.executors.indexers.keyvalue import BinaryPbIndexer
from jina.executors.indexers import CompoundIndexer

a = NumpyIndexer(index_filename='vec.gz')
b = BinaryPbIndexer(index_filename='vec.gz')
c = CompoundIndexer()
c.components = lambda: [a, b]

Compose Flow from YAML

Now let's add our indexer YAML file to the Flow with .add(uses=). Let's also add two shards to the indexer to improve its scalability:

f = Flow().add(uses='MyEncoder', parallel=2).add(uses='myindexer.yml', shards=2).plot()

When you have many arguments, constructing a Flow in Python can get cumbersome. In that case, you can simply move all arguments into one flow.yml:

!Flow
version: '1.0'
pods:
  - name: encode
    uses: MyEncoder
    parallel: 2
  - name:index
    uses: myindexer.yml
    shards: 2

And then load it in Python:

f = Flow.load_config('flow.yml')

Search

Querying a Flow is similar to what we did with indexing. Simply load the query Flow and switch from f.index to f.search. Say you want to retrieve the top 50 documents that are similar to your query and then plot them in HTML:

f = Flow.load_config('flows/query.yml')
with f:
    f.search_ndarray(numpy.random.random([10, 28, 28]), shuffle=True, on_done=plot_in_html, top_k=50)

Evaluation

To compute precision recall on the retrieved result, you can add _eval_pr, a built-in evaluator for computing precision & recall.

f = (Flow().add(...)
           .add(uses='_eval_pr'))

You can construct an iterator of query and groundtruth pairs and feed to the flow f, via:

from jina import Document

def query_generator():
    for _ in range(10):
        q = Document()
        # now construct expect matches as groundtruth
        gt = Document(q, copy=True)  # make sure 'gt' is identical to 'q'
        gt.matches.append(...)
        yield q, gt

f.search(query_iterator, ...)

REST Interface

In practice, the query Flow and the client (i.e. data sender) are often physically separated. Moreover, the client may prefer to use a REST API rather than gRPC when querying. You can set port_expose to a public port and turn on REST support with restful=True:

f = Flow(port_expose=45678, restful=True)

with f:
    f.block()

That is the essence behind jina hello-world. It is merely a taste of what Jina can do. We’re really excited to see what you do with Jina! You can easily create a Jina project from templates with one terminal command:

pip install jina[hub] && jina hub new --type app

This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Learn

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited   

Jina 101: First Things to Learn About Jina

  English日本語FrançaisPortuguêsDeutschРусский язык中文عربية

Examples (View all)

Example code to build your own projects

📄

Semantic Wikipedia Search with Transformers and DistilBERT

Brand new to neural search? See a simple text-search example to understand how Jina works

📄

Add Incremental Indexing to Wikipedia Search

Index more effectively by adding incremental indexing to your Wikipedia search

📄

Build a NLP Semantic Search System with Transformers

Upgrade from plain search to sentence search and practice your Flows and Pods by searching sentences from Wikipedia

📄

Search Lyrics with Transformers and PyTorch

Get a better understanding of chunks by searching a lyrics database. Now with shiny front-end!

🖼️

Google's Big Transfer Model in (Poké-)Production

Use SOTA visual representation for searching Pokémon!

🎧

Search YouTube audio data with Vggish

A demo of neural search for audio data based Vggish model.

🎞️

Search Tumblr GIFs with KerasEncoder

Use prefetching and sharding to improve the performance of your index and query flow when searching animated GIFs.

Please check our examples repo for advanced and community-submitted examples.

Want to read more? Check our Founder Han Xiao's blog and our official blog.

Documentation

Apart from the learning resources we provided above, We highly recommended you go through our documentation to master Jina.

Our docs are built on every push, merge, and release of Jina's master branch. Documentation for older versions is archived here.

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors

All Contributors

Community

  • Code of conduct - play nicely with the Jina community
  • Slack workspace - join #general on our Slack to meet the team and ask questions
  • YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow and interact with us using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020-2021 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Issues
  • Text-by-Audio

    Text-by-Audio

    Hi,

    I want to retrieve text by searching for an audio using AudioClip model.

    First, I created indexing of text (car-horn, coughing, alarm-clock, thunderstorm etc) and using AudioClipTextEncoder for embedding.

    After that I am searching for an audio where i am using AudioClipEncoder for embedding.

    For both text and audio indexing simple-indexer is used.

    While searching for an audio i created chunks using segmenter and using MyRanker for ranking scores where i modified some script.

    The output is: input_audio: AudioCLIP/demo/audio/thunder_3-144891-B-19.wav, len-of-chunks:1 +- id: e5eceb26737311ec83fd2fdc0fc83b8e score: 0.931110, cat input_audio: AudioCLIP/demo/audio/coughing_1-58792-A-24.wav, len-of-chunks:1 +- id: e5ecf968737311ec83fd2fdc0fc83b8e score: 0.923869, cat input_audio: AudioCLIP/demo/audio/cat_3-95694-A-5.wav, len-of-chunks:1 +- id: e5ed00de737311ec83fd2fdc0fc83b8e score: 0.845475, cat input_audio: AudioCLIP/demo/audio/car_horn_1-24074-A-43.wav, len-of-chunks:1 +- id: e5ed0778737311ec83fd2fdc0fc83b8e score: 0.928283, cat input_audio: AudioCLIP/demo/audio/alarm_clock_3-120526-B-37.wav, len-of-chunks:1 +- id: e5ed0dcc737311ec83fd2fdc0fc83b8e score: 0.919222, alarm_clock

    As above for 5 inputs only 2 outputs are correct. Please let me know why is that so ? Since AudioClipEncoder is giving correct outputs for all audios.

    opened by snaaz21 55
  • [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    Describe your proposal/problem

    Dear Jina Team,

    Recently, I tried nlp-simple example and replaced TransformerTorchEncoder with TextPaddlehubEncoder. However, I got the following error.

    Traceback (most recent call last):
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 73, in _load_executor
        self._executor = BaseExecutor.load_config(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 531, in load_config
        return JAML.load(tag_yml, substitute=False)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 89, in load
        r = yaml.load(stream, Loader=JinaLoader)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
        return loader.get_single_data()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
        return self.construct_document(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
        data = self.construct_object(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
        data = constructor(self, node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 422, in _from_yaml
        return get_parser(cls, version=data.get('version', None)).parse(cls, data)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/parsers/executor/legacy.py", line 130, in parse
        obj = cls(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 82, in __call__
        getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 174, in _post_init_wrapper
        self.post_init()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/hub/encoders/nlp/TextPaddlehubEncoder/__init__.py", line 38, in post_init
        self.model = hub.Module(name=self.model_name)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 171, in __new__
        module = cls.init_with_name(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 263, in init_with_name
        user_module_cls = manager.install(name=name, version=version, source=source, update=update, branch=branch)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 188, in install
        return self._install_from_name(name, version)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 263, in _install_from_name
        return self._install_from_url(item['url'])
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 256, in _install_from_url
        return self._install_from_archive(file)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 361, in _install_from_archive
        return self._install_from_directory(directory)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 345, in _install_from_directory
        hub_module_cls = HubModule.load(self._get_normalized_path(module_info.name))
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 219, in load
        paddle.set_device(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/device.py", line 166, in set_device
        framework._set_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 317, in _set_expected_place
        _set_dygraph_tracer_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 311, in _set_dygraph_tracer_expected_place
        _dygraph_tracer_._expected_place = place
    OSError: (External)  Cuda error(3), initialization error.
      [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:229)
    

    It seems to be related with https://github.com/PaddlePaddle/Paddle/issues/25185.

    However, I have no idea how to fix this. Could you help me to figure it out? Thank you very much!


    Environment

    paddlehub==2.0.0 paddlenlp==2.0.1 paddlepaddle-gpu==2.0.2.post90 jina==1.3.0

    Screenshots

    image

    area/core type/bug 
    opened by ryh95 47
  • No documents are found with BinaryPbIndexer in 1.1.2

    No documents are found with BinaryPbIndexer in 1.1.2

    Versions 1.0.11: everything works as expected 1.1.2: BinaryPbIndexer doesn't find any documents

    Detailed description He Jina team, when I upgrade to 1.1.2 and reindex my dataset, the BinaryPbIndexer no longer works. It doesn't raise any errors, but I just can't find any documents. Just as a remark, I use custom ids which always consist of 33 characters. Where there any changes for the BinaryPbIndexer, the KVIndexDriver or the KVSearchDriver?

    area/core priority/critical type/bug status/done 
    opened by janandreschweiger 39
  • Permanently delete documents from storage

    Permanently delete documents from storage

    Hey Jina team!

    As described here documents persist the deletion and are only marked as inaccessible.

    My question: Is there a possibility to delete documents from storage without reindexing everything again? Deletion performance isn't an issue for us, but reindexing everything is very inconvenient in our use case.

    Thanks! I really appreciate your kind support.

    Sub-tasks

    • [x] Delete from NpIndexer. @ #2046
    • [x] Delete from BinaryPbIndexer. @ https://github.com/jina-ai/jina/pull/2102
    area/core priority/important-soon type/feature-request status/done 
    opened by ace-kay-law-neo 39
  • File seems to be closed when trying to index

    File seems to be closed when trying to index

    Yes @JoanFM, using the prefetch argument in the flow-config solved the original problem. I tried to pass the batch_size and the request_size like you, but I still get the same error. I have some additional logs, which might indicate what causes the problem. As I said, the first few batches are indexed successfully, but at some point I get this error:

            [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            [email protected][I]:#sent: 3 #recv: 4 sent_size: 13.8 MB recv_size: 1.2 MB
          [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸⚐
            [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸⚐
       [email protected][S]:artifacts of this executor (wrap_idx) is persisted to /var/huforce/fs/workspace/wrap_idx-0/wrap_idx.bin
    [email protected][S]:artifacts of this executor (chunk_idx) is persisted to /var/huforce/fs/workspace/chunk_idx-0/chunk_idx.bin
          [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
          [email protected][I]:#sent: 3 #recv: 4 sent_size: 1.9 MB recv_size: 15.6 MB
            [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸doc_idx/ZEDRuntime▸⚐
    DEBUG | Message: Successfully indexed 4 documents!
            [email protected][S]:artifacts of this executor (doc_idx_file) is persisted to /var/huforce/fs/workspace/doc_idx_file-0/doc_idx_file.bin
            [email protected][I]:#sent: 3 #recv: 4 sent_size: 6.3 KB recv_size: 2.2 MB
           | 📃     16 ⏱️ 262.9s 🐎 0.1/s      4   requests        [email protected][I]:send: 5 recv: 4 pending: 1
            [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸⚐
    DEBUG | Message: File-Info: research_admission.txt
    DEBUG | Message: File-Info: cloudcomputing.pptx
    DEBUG | Message: File-Info: WS_2012_White_Paper_Hyper-V.pdf
    DEBUG | Message: File-Info: writing-sql-queries.docx
            [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            [email protected][I]:#sent: 5 #recv: 6 sent_size: 1.2 MB recv_size: 5.2 KB
            [email protected][I]:#sent: 4 #recv: 5 sent_size: 15.6 MB recv_size: 1.2 MB
          [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime▸⚐
            [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸⚐
          [email protected][I]:#sent: 4 #recv: 5 sent_size: 2.2 MB recv_size: 16.1 MB
    DEBUG | Message: Encoding 552 paragraphs
          [email protected][E]:ValueError('write to closed file')
     add "--quiet-error" to suppress the exception details
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 198, in _msg_callback
        self._zmqlet.send_message(self._callback(msg))
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 184, in _callback
        self._pre_hook(msg)._handle(msg)._post_hook(msg)
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 177, in _handle
        self._executor(self.request_type)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/__init__.py", line 595, in __call__
        d()
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/__init__.py", line 381, in __call__
        self._apply_all(documents, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/index.py", line 58, in _apply_all
        self.exec_fn(keys, values)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/decorators.py", line 44, in arg_wrapper
        f = func(self, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/indexers/keyvalue.py", line 178, in add
        self.write_handler.header.write(
    ValueError: write to closed file
            [email protected][I]:recv IndexRequest  from gateway▸crafter/ZEDRuntime▸encoder/ZEDRuntime▸chunk_idx/ZEDRuntime✖▸doc_idx/ZEDRuntime▸⚐
    dev_ai            | DEBUG | Message: Successfully indexed 4 documents!
    

    What's also strange is, that the failed documents for the chunk-idx are passed on to the doc-idx, as you can see in the logs.

    My flow for indexing:

    !AsyncFlow
    with:
      logserver: false
      show-exc-info: true
      prefetch: 2
      # request-size: 2
      # batch_size: 2
    pods:
      crafter:
        uses: pods/craft.yml
      encoder:
        uses: pods/encode.yml
        timeout_ready: 600000
      chunk_idx:
        uses: pods/chunk.yml
      doc_idx:
        uses: pods/doc.yml
    

    Originally posted by @nero-nazok in https://github.com/jina-ai/jina/issues/2298#issuecomment-817084516

    area/core type/bug 
    opened by JoanFM 32
  • DOCQA..

    DOCQA..

    I am trying to parse a pdf & apply the DOCQA model to it.Roberta as well as BERT..!! But I am not getting right answer for it.. whereas independently it's works accurately... !!

    This is my flow.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
    

    This is my query.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
      - name: generator
        uses: Generator
        py_modules: "flows/generator_roberta.py"
    
    opened by jyotikhetan 29
  • Gateway error: received an empty stream from the client

    Gateway error: received an empty stream from the client

    Description

    I encountered an error in flow.post method while trying to run python app.py -t index in multires-lyrics-search after cloning this repository: examples. The error message was: [email protected][E]:receive an empty stream from the client! please check your client's inputs, you can use "Client.check_input(inputs)".


    Environment

    • jina 2.0.18
    • jina-proto 0.0.85
    • jina-vcs-tag (unset)
    • libzmq 4.3.4
    • pyzmq 1.21.5
    • protobuf 3.20.0
    • proto-backend cpp
    • grpcio 1.44.0
    • pyyaml 6.0
    • python 3.7.13
    • platform Linux
    • platform-release 5.10.105-1-MANJARO
    • platform-version #1 SMP PREEMPT Fri Mar 11 14:12:33 UTC 2022
    • architecture x86_64
    • processor
    • uid 5025763201072
    • session-id 7d472a12-b722-11ec-a806-049226d49030
    • uptime 2022-04-08T15:58:49.863145
    • ci-vendor (unset)
    • JINA_ARRAY_QUANT (unset)
    • JINA_CONTROL_PORT (unset)
    • JINA_DEFAULT_HOST (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_HUBBLE_REGISTRY (unset)
    • JINA_HUB_CACHE_DIR (unset)
    • JINA_HUB_ROOT (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_ID (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_LOG_WORKSPACE (unset)
    • JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
    • JINA_POD_NAME (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_VCS_VERSION (unset)
    • JINA_MP_START_METHOD (unset)
    opened by Jackal1586 28
  • feat: remove head for non sharded deployments

    feat: remove head for non sharded deployments

    Remove heads for non sharded Deployments

    Description

    This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers. Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

    This is a breaking change :boom: It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

    Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

    • [x] Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD
    • [x] Set all worker Pods in the Gateway (Outside K8s)
    • [x] Make the gateway send to worker Pods
    • [x] Dont create heads for Deployments with no shards
    • [x] Implement reducing after multiple needs in the Gateway
    • [x] Adapt tests
    • [x] Change docs

    Closes #4503 (issue)

    area/core area/helper area/testing size/XL area/docs area/cli area/daemon 
    opened by jacobowitz 27
  • Unable to run Jina on Raspberry Pi 4

    Unable to run Jina on Raspberry Pi 4

    I installed Jina using the installation instructions in the docs and I was unable to run even the Jina command without the error

    free(): invalid pointer
    Aborted
    

    I have tried installing Jina from master branch, 1.2.4, and from pip. Yet, the problem persists


    Environment Can't actually run this command.

    Latest version of Raspbian OS (aarm64), on a RPi4. Python 3.7.3

    Screenshots

    type/bug 
    opened by navanchauhan 25
  • Server error popping up while running jina backend

    Server error popping up while running jina backend

    While running the backend file of Jina this file Link

    TimeoutError('no response from sock tcp://0.0.0.0:60797 after timeout=5000ms, please check the following: is the server still online? is the network broken? are "port" correct? ') during <bound method BasePea.close of <jina.peapods.peas.BasePea object at 0x7f052ce70430>> add "--quiet-error" to suppress the exception details


    Environment

    • jina 2.2.0
    • jina-proto 0.0.86
    • jina-vcs-tag (unset)
    • libzmq 4.3.4
    • pyzmq 1.19.5
    • protobuf 3.19.3
    • proto-backend cpp
    • grpcio 1.43.0
    • pyyaml 5.3.1
    • python 3.8.10
    • platform Linux
    • platform-release 4.4.0-19041-Microsoft
    • platform-version #1237-Microsoft Sat Sep 11 14:32:00 PST 2021
    • architecture x86_64
    • processor x86_64
    • uid 4459196994945
    • session-id 389ff0fc-7ba8-11ec-83b5-040e3cda5581
    • uptime 2022-01-22T22:54:57.138560
    • ci-vendor (unset)
    • JINA_ARRAY_QUANT (unset)
    • JINA_CONTROL_PORT (unset)
    • JINA_DEFAULT_HOST (unset)
    • JINA_DEFAULT_TIMEOUT_CTRL (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_HUBBLE_REGISTRY (unset)
    • JINA_HUB_CACHE_DIR (unset)
    • JINA_HUB_ROOT (unset)
    • JINA_K8S_USE_TEST_PIP (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_ID (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_LOG_WORKSPACE (unset)
    • JINA_MP_START_METHOD (unset)
    • JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
    • JINA_POD_NAME (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_VCS_VERSION (unset) Screenshots image

    image

    image

    image

    area/community 
    opened by Vidit-Ostwal 24
  • `As-a-Service in 10 Extra Lines ` : aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe

    `As-a-Service in 10 Extra Lines ` : aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe

    Following the indications here: https://github.com/jina-ai/jina#as-a-service-in-10-extra-lines-

    I'm trying to execute the following script:

    # https://github.com/jina-ai/jina#as-a-service-in-10-extra-lines-
    
    from jina import Document, DocumentArray, Executor, Flow, requests
    
    # pre-processing step
    class PreprocImg(Executor):
        @requests
        def foo(self, docs: DocumentArray, **kwargs):
            for d in docs:
                (d.load_uri_to_image_blob()  # load
                 .set_image_blob_normalization()  # normalize color
                 .set_image_blob_channel_axis(-1, 0))  # switch color axis
    
    
    # embedding step
    class EmbedImg(Executor):
        def __init__(self, **kwargs):
            super().__init__(**kwargs)
            import torchvision
            self.model = torchvision.models.resnet50(pretrained=True)        
    
        @requests
        def foo(self, docs: DocumentArray, **kwargs):
            docs.embed(self.model)
    
    # matching step
    class MatchImg(Executor):
        _da = DocumentArray()
    
        @requests(on='/index')
        def index(self, docs: DocumentArray, **kwargs):
            self._da.extend(docs)
    
        @requests(on='/search')
        def foo(self, docs: DocumentArray, **kwargs):
            docs.match(self._da)
            for d in docs.traverse_flat('r,m'):  # only require for visualization
                d.convert_uri_to_datauri()  # convert to datauri
                d.pop('embedding', 'blob')  # remove unnecessary fields for save bandwidth
    
    # Connect all Executors in a Flow, scale embedding to 3
    f = Flow(port_expose=12345, protocol='http').add(uses=PreprocImg).add(uses=EmbedImg, replicas=3).add(uses=MatchImg)
    
    # Index image data and serve REST query publicly
    with f:
        f.post('/index', DocumentArray.from_files('img/*.jpg'), show_progress=True, request_size=8)
        f.block()
    

    But I get these errors:

    (vespa) [email protected]:~/vespa-jina/jina-sample-apps$ python3 AsAService-ResNet501ImageSearch.py
               [email protected][I]:🎉 Flow is ready to use!                                                  
        🔗 Protocol: 		HTTP
        🏠 Local access:	0.0.0.0:12345
        🔒 Private network:	192.168.1.7:12345
        🌐 Public address:	109.116.161.240:12345
        💬 Swagger UI:		http://localhost:12345/docs
        📚 Redoc:		http://localhost:12345/redoc
    UserWarning: ignored unknown argument: ['12345']. (raised from /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/helper.py:691)
    ⠋ Working... ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:03 estimating... UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:189.) (raised from /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/docarray/ndarray/__init__.py:359)
    ⠼      ERROR ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:05:00 0% ETA: 1 day and 7 hours and 15 minutes and 13 seconds Task exception was never retrieved
    future: <Task finished name='Task-302' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ClientOSError(32, 'Broken pipe')>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
        message, payload = await protocol.read()  # type: ignore[union-attr]
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
        await self._waiter
    aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe
    Task exception was never retrieved
    future: <Task finished name='Task-78' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-169' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-166' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-163' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-75' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-177' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-72' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-157' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-154' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-69' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-150' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-147' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-66' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-144' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-141' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-138' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-136' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-180' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-172' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-175' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-81' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-84' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-160' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-133' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-130' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-127' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-124' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-121' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-118' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-115' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-112' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-109' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-106' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-103' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-99' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-96' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-206' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-204' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-93' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-201' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-199' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-87' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-90' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-192' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-189' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-183' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-186' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=TimeoutError()>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 913, in start
        self._continue = None
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/helpers.py", line 721, in __exit__
        raise asyncio.TimeoutError from None
    asyncio.exceptions.TimeoutError
    Task exception was never retrieved
    future: <Task finished name='Task-300' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ServerDisconnectedError('Server disconnected')>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
        message, payload = await protocol.read()  # type: ignore[union-attr]
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
        await self._waiter
    aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
    Task exception was never retrieved
    future: <Task finished name='Task-304' coro=<HTTPClientlet.send_message() done, defined at /home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py:71> exception=ServerDisconnectedError('Server disconnected')>
    Traceback (most recent call last):
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/jina/clients/base/helper.py", line 80, in send_message
        return await self.session.post(url=self.url, json=req_dict).__aenter__()
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 1138, in __aenter__
        self._resp = await self._coro
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client.py", line 559, in _request
        await resp.start(conn)
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 898, in start
        message, payload = await protocol.read()  # type: ignore[union-attr]
      File "/home/raphy/anaconda3/envs/vespa/lib/python3.8/site-packages/aiohttp/streams.py", line 616, in read
        await self._waiter
    aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
    

    OS: Ubuntu 20.04 python version : Python 3.8.12

    How to solve the problem?

    priority/critical 
    opened by raphael10-collab 22
  • feat(websocket): http healthcheck endpoint

    feat(websocket): http healthcheck endpoint

    Goal

    Even though AWS ALB supports Websocket traffic, it can't do health checks for WebSocket. It can do health checks with HTTP protocol though - I'm enabling the same to add WebSocket gateway support in JCloud. Now, it throws bunch of 404s since the endpoint actually doesn't exist.

    This PR adds an HTTP endpoint GET / for WebsocketGateway like for HTTPGateway (since both are FastAPI apps on Uvicorn server). I'm not sure whether this would help other load balancers, though.

    • [ ] check and update documentation. See guide and ask the team.
    area/core size/S area/testing 
    opened by deepankarm 2
  • Rewrite k8s docs

    Rewrite k8s docs

    • [ ] mention image build process and link to corresponding doc section
    • [ ] config.yml is hard coded in the k8s executor yaml, warn about this
    • [ ] Specify that our reliance on service meshes is a k8s+gRPC "problem", not one that we introduced
    opened by JohannesMessner 0
  • Docs: explain error modelling and failures

    Docs: explain error modelling and failures

    Explain how (connection) errors are handled, locally, in docker and k8s, and how errors are reported to the user. Also explain that retrying in service mesh is potentially dangerous because we also retry.

    ToDo:

    • [ ] error retry mechanism
      • [ ] in-house solution
      • [ ] k8s and service mesh
    opened by JohannesMessner 0
Releases(v3.4.9)
  • v3.4.9(May 26, 2022)

    Release Note (3.4.9)

    Release time: 2022-05-26 11:40:41

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [674e8121] - success box ui (Han Xiao)

    🍹 Other Improvements

    • [a2b5e304] - Revert "ci: add create-release.yml (#4759)" (#4835) (Han Xiao)
    • [f053d764] - version: the next version will be 3.4.9 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.6(May 23, 2022)

    Release Note (3.4.6)

    Release time: 2022-05-23 19:21:36

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [049666d9] - code review (#4821) (Han Xiao)
    • [688370db] - dockerfile: install security updates in docker image (Han Xiao)

    🍹 Other Improvements

    • [174dbde1] - add security analysis (Han Xiao)
    • [93db296a] - version: the next version will be 3.4.6 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.5(May 23, 2022)

    Release Note (3.4.5)

    Release time: 2022-05-23 12:00:14

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, Johannes Messner, ZiniuYu, Joan Fontanals, Zhaofeng Miao, Jina Dev Bot, 🙇

    🆕 New Features

    • [9c1fda96] - executor: add self logger to executor (#4818) (Han Xiao)
    • [e553d913] - smart replica retry (#4794) (Johannes Messner)
    • [a9ed27ed] - add additional metrics (#4789) (samsja)
    • [c88d2cf5] - hubble: use fixed domain for hubble api (#4804) (Zhaofeng Miao)

    🐞 Bug fixes

    • [aa2d7c8d] - dockerfile: install security updates in docker image (Han Xiao)
    • [87de3c09] - define metrics in worker before grpc server (#4816) (samsja)
    • [a2bc2302] - avoid closing loop at fork time (#4803) (Joan Fontanals)
    • [3fb3b419] - warn user when calling load config from flow instance (#4787) (samsja)

    🧼 Code Refactoring

    • [93f06fc5] - simplify event loop usage (#4811) (Joan Fontanals)

    📗 Documentation

    • [3a7d95b6] - add specification of executor and flow yaml (#4795) (Johannes Messner)
    • [10ab59f1] - fix Flow docs typo (#4815) (ZiniuYu)
    • [15d83ba8] - round robin gpu scheduling (#4805) (samsja)

    🍹 Other Improvements

    • [94153117] - docs: update TOC (Jina Dev Bot)
    • [3c7d0b3e] - version: the next version will be 3.4.5 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.4(May 18, 2022)

    Release Note (3.4.4)

    Release time: 2022-05-18 12:54:06

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Delgermurun, Sha Zhou, Jina Dev Bot, Roshan Jossy, 🙇

    🐞 Bug fixes

    • [0372f10f] - unsetenv not existing on Windows (#4800) (Joan Fontanals)
    • [99ccff1f] - remove executor secret from flow mermaid chart (#4801) (Delgermurun)
    • [d66e98c1] - qabot: hide the slot elements on the docs page (#4799) (Sha Zhou)

    📗 Documentation

    • [f0f5948e] - tracking: remove adding utm_source to links (#4798) (Roshan Jossy)

    🍹 Other Improvements

    • [32d85a50] - docs: update TOC (Jina Dev Bot)
    • [2b5efac1] - version: the next version will be 3.4.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.3(May 17, 2022)

    Release Note (3.4.3)

    Release time: 2022-05-17 10:52:53

    🙇 We'd like to thank all contributors for this new release! In particular, samsja, Johannes Messner, Jina Dev Bot, Han Xiao, 🙇

    🐞 Bug fixes

    • [a32bfe21] - pop the init time instead of normal access (#4791) (samsja)
    • [c663e291] - add check that jaml parses to correct class (#4790) (Johannes Messner)

    🍹 Other Improvements

    • [ec923d02] - docs: update TOC (Jina Dev Bot)
    • [95c1519d] - fix readme (Han Xiao)
    • [6194244f] - version: the next version will be 3.4.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.2(May 13, 2022)

    Release Note (3.4.2)

    Release time: 2022-05-13 20:09:02

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🆕 New Features

    • [1dd09184] - deployment: add roundrobin cuda device assignment (#4786) (Han Xiao)

    🍹 Other Improvements

    • [8c4d1761] - remove unnecessary steps on chore commit (Han Xiao)
    • [01503bcd] - version: the next version will be 3.4.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.1(May 13, 2022)

    Release Note (3.4.1)

    Release time: 2022-05-13 11:55:54

    🙇 We'd like to thank all contributors for this new release! In particular, Johannes Messner, Delgermurun, Han Xiao, Roshan Jossy, Florian Hönicke, Jina Dev Bot, samsja, 🙇

    🆕 New Features

    • [072a47a4] - better error messages when gateway can't connect to other deployment (#4677) (Johannes Messner)
    • [6514ca95] - hubble: executor push/pull user integration (#4770) (Delgermurun)

    🐞 Bug fixes

    • [fb529399] - sandbox: send secret in the request (#4781) (Delgermurun)
    • [2e9484ef] - yaml jtype flow (#4776) (samsja)

    🧼 Code Refactoring

    • [934ecd8f] - flow: improve success box (#4782) (Han Xiao)
    • [b0df40ae] - logging no print (#4777) (Florian Hönicke)

    📗 Documentation

    • [ad5ed0dc] - tracking: fix utm source (#4780) (Roshan Jossy)
    • [7de44506] - add jina now logo (#4775) (Florian Hönicke)

    🏁 Unit Test and CICD

    • [d7c48d5f] - hubble: fix failing test (#4783) (Delgermurun)

    🍹 Other Improvements

    • [ec136dd8] - update announcement in readme and docs (Jina Dev Bot)
    • [13a954be] - docs: update TOC (Jina Dev Bot)
    • [36ca89d3] - version: the next version will be 3.4.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.4.0(May 10, 2022)

    Highlights :star2:

    Monitoring :computer:

    Now Jina enables you to monitor the performance of your Executors and the complete Flow with the help of Prometheus and allows you to build a Grafana dashboard to gain better insights on how your system works.

    from jina import Flow
    
    f = Flow(monitoring=True).add().add()
    
    with f:
    	f.block()
    

    Screenshot from 2022-05-10 11-05-21

    #4636 #4635 #4638 #4672 #4689 #4687 #4735 #4738 #4737 #4758 #4752 #4768

    Robustness :muscle:

    We have improved the robustness of the system and the network communication between microservices in a Flow and have put a lot of work on getting better information of potential failures in the system.

    Changes include:

    • Adding a timeout parameter for the Gateway for its grpc communication with Executors #4660
    • Ensuring a graceful shutdown of Runtimes in Kubernetes to avoid requests loss #4682
    • Adding an extensive test for handling Pod failues in Kubernetes #4743
    • Better error reporting of errors happening in the Head of a sharded Executor #4640
    • Better error reporting when Flow fails to start #4652

    UI improvements :haircut:

    Jina comes with a better UI and a better logging system to help the user get better information from the logs in the best format based on rich. Screenshot from 2022-05-10 11-09-03

    #4614 #4697 #4724 #4736 #4739

    Other changes

    • Remove unnecessary argument from Executors runtime_backend (#4644)
    • Pass environment variables to Gateway and Head Runtimes (#4646)
    • Add resource layer with gpu in docker-compose files when to_docker_compose is called for Services requiring it (#4718)
    • You can have inmutable tags when pushing Executors into the Hub (#4676)
    • target_executor parameter is considered a regex pattern and not a exact string. All Executors matching this regex will receive the request. (#4756)
    • Gateway only sends request to Executor if the endpoint is exposed by it (#4756)

    Bug fixes :bug:

    • Avoid leaking file descriptors when client sends requests (#4734)
    • Fix flow plotting when private Executor is used (#4643)
    • Executor could not start processes internally (#4692)
    • Deadlock on Executor start due to ModuleLock error (#4693)
    • Flow early stop when startup failure (#4726)

    Breaking changes :warning:

    target_executor is treated as a regex pattern and not as a string.

    Before:

    from docarray import DocumentArray
    from jina import Flow, Executor, requests
    
    
    class Exec(Executor):
    
       @requests
       def foo(self, docs, **kwargs):
           for doc in docs:
               doc.text += 'calling foo '
    
    
    f = Flow().add(name='exec', uses=Exec).add(name='exec_after', uses=Exec)
    
    with f:
       res = f.post(on='/', inputs=DocumentArray.empty(1), target_executor='exec')
       print(res.texts)
    

    Before, it would give:

    ['calling foo ']
    

    Now:

    ['calling foo calling foo ']
    

    :heart: Thanks to the community for all the support and hard work!

    Source code(tar.gz)
    Source code(zip)
  • v3.3.25(May 5, 2022)

    Release Note (3.3.25)

    Release time: 2022-05-05 17:39:33

    🙇 We'd like to thank all contributors for this new release! In particular, Tobias Jacobowitz, Roshan Jossy, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [c1f0ae51] - close loop from run_async (#4734) (Tobias Jacobowitz)

    📗 Documentation

    • [36ce8359] - tracking: update refer links to other docs with utm_source (#4744) (Roshan Jossy)

    🍹 Other Improvements

    • [913d1f31] - docs: update TOC (Jina Dev Bot)
    • [d5c9967c] - version: the next version will be 3.3.25 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.24(May 4, 2022)

    Release Note (3.3.24)

    Release time: 2022-05-04 14:52:39

    🙇 We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, samsja, Jina Dev Bot, 🙇

    🆕 New Features

    • [f9f86004] - inherit monitoring arg from Flow (#4738) (samsja)

    🍹 Other Improvements

    • [58847187] - grpc version>=1.46.0 (#4746) (Deepankar Mahapatro)
    • [e3c45bf5] - docs: update TOC (Jina Dev Bot)
    • [c4175f95] - version: the next version will be 3.3.24 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.23(May 2, 2022)

    Release Note (3.3.23)

    Release time: 2022-05-02 20:10:11

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [de381ba0] - logger: make log format more compact (#4739) (Han Xiao)
    • [944afbc5] - move prometheus depedency to perf (#4735) (samsja)
    • [3a1a8f64] - rich console logs (#4724) (samsja)

    🍹 Other Improvements

    • [4d1e4685] - fix rich log format string (#4736) (Han Xiao)
    • [1ddc3953] - fix rich log format string (Han Xiao)
    • [f8b994dc] - docs: update TOC (Jina Dev Bot)
    • [2f08c1b6] - version: the next version will be 3.3.23 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.22(Apr 30, 2022)

    Release Note (3.3.22)

    Release time: 2022-04-30 18:43:18

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [1516c276] - flow: early stop (#4726) (Han Xiao)

    🍹 Other Improvements

    • [930d4bab] - remove redundant colored (Han Xiao)
    • [5152c983] - docs: update TOC (Jina Dev Bot)
    • [bca843ad] - version: the next version will be 3.3.22 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.21(Apr 29, 2022)

    Release Note (3.3.21)

    Release time: 2022-04-29 19:17:10

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🍹 Other Improvements

    • [2b2b2e61] - remove redundant code (Han Xiao)
    • [5ad319ef] - version: the next version will be 3.3.21 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.20(Apr 29, 2022)

    Release Note (3.3.20)

    Release time: 2022-04-29 13:48:42

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Jina Dev Bot, 🙇

    🆕 New Features

    • [55d1f44e] - allow jina now to unset jina_check_version (#4722) (Joan Fontanals)
    • [615e3efe] - monitoring support for k8s (#4687) (samsja)

    🐞 Bug fixes

    • [976792e9] - edit conda recipe generator (#4723) (Joan Fontanals)

    🍹 Other Improvements

    • [f5dacb1c] - update announcement in readme and docs (Jina Dev Bot)
    • [16b6f154] - docs: update TOC (Jina Dev Bot)
    • [2ff693f3] - version: the next version will be 3.3.20 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.19(Apr 28, 2022)

    Release Note (3.3.19)

    Release time: 2022-04-28 16:14:00

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Yanlong Wang, Jina Dev Bot, 🙇

    🆕 New Features

    • [93bda042] - allow user not to print version check print (#4720) (Joan Fontanals)
    • [19e5cd7e] - immutable tags for executors (#4676) (Yanlong Wang)

    🐞 Bug fixes

    • [ac204dcb] - rename immutable-tag to protected-tag (#4721) (Yanlong Wang)

    🍹 Other Improvements

    • [a884f616] - version: the next version will be 3.3.19 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.18(Apr 28, 2022)

    Release Note (3.3.18)

    Release time: 2022-04-28 13:17:43

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Johannes Messner, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [0def13df] - add resource layer in docker compose gpu (#4718) (Joan Fontanals)
    • [e57f9524] - remove version check from cli (#4716) (Joan Fontanals)
    • [07ae316d] - fix grpcio-reflection version (#4715) (Joan Fontanals)
    • [917604de] - update failing test (#4711) (samsja)
    • [59340e05] - the cd (#4714) (Johannes Messner)

    🍹 Other Improvements

    • [e75efaef] - docs: update TOC (Jina Dev Bot)
    • [87b4b035] - version: the next version will be 3.3.18 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.17(Apr 27, 2022)

    Release Note (3.3.17)

    Release time: 2022-04-27 21:10:33

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

    🍹 Other Improvements

    • [9f83ab6b] - readme badge (Han Xiao)
    • [b2e11974] - sort and optimize imports (#4713) (Han Xiao)
    • [22d5ce67] - update readme (Han Xiao)
    • [d2423b27] - version: the next version will be 3.3.17 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.16(Apr 27, 2022)

    Release Note (3.3.16)

    Release time: 2022-04-27 19:38:37

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Joan Fontanals, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [4bafca88] - logger: use docker log handler for container (#4712) (Han Xiao)
    • [bb227615] - fix add deploy resources when gpus detected (#4706) (Joan Fontanals)

    🍹 Other Improvements

    • [955adb40] - version: the next version will be 3.3.16 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.15(Apr 27, 2022)

    Release Note (3.3.15)

    Release time: 2022-04-27 15:42:23

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Han Xiao, Jina Dev Bot, 🙇

    🧼 Code Refactoring

    • [a8f50dd0] - change way of importing in cli (#4709) (Joan Fontanals)
    • [734edbc2] - some UI improvement (#4705) (Han Xiao)

    🍹 Other Improvements

    • [b1bff161] - version: the next version will be 3.3.15 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.14(Apr 27, 2022)

    Release Note (3.3.14)

    Release time: 2022-04-27 10:50:31

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [9a579cd6] - use warnings and threading (#4703) (Joan Fontanals)

    🍹 Other Improvements

    • [06fc848c] - version: the next version will be 3.3.14 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.13(Apr 27, 2022)

    Release Note (3.3.13)

    Release time: 2022-04-27 09:08:20

    🙇 We'd like to thank all contributors for this new release! In particular, felix-wang, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [e56d787c] - parsing requirements of executor (#4698) (felix-wang)

    🍹 Other Improvements

    • [d1c44920] - version: the next version will be 3.3.13 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.12(Apr 27, 2022)

    Release Note (3.3.12)

    Release time: 2022-04-27 08:57:34

    🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Yanlong Wang, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [a3b71c72] - parsers: clearify flow args (#4701) (Han Xiao)
    • [babd97d0] - hubio: improve error message from hubble (#4674) (Yanlong Wang)

    🍹 Other Improvements

    • [524bedd5] - docs: update TOC (Jina Dev Bot)
    • [7649d5d9] - version: the next version will be 3.3.12 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.11(Apr 26, 2022)

    Release Note (3.3.11)

    Release time: 2022-04-26 14:47:52

    🙇 We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, Tobias Jacobowitz, Jina Dev Bot, Joan Fontanals, 🙇

    🆕 New Features

    • [25b8df17] - graceful shutdown (#4682) (Tobias Jacobowitz)

    🐞 Bug fixes

    • [e23ff11a] - pod does not start runtime process as daemon (#4692) (Joan Fontanals)

    🍹 Other Improvements

    • [05222069] - disable console color with envvar (#4697) (Deepankar Mahapatro)
    • [655213d0] - docs: update TOC (Jina Dev Bot)
    • [d9b5b718] - version: the next version will be 3.3.11 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.10(Apr 25, 2022)

    Release Note (3.3.10)

    Release time: 2022-04-25 21:00:48

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [5af4efe3] - fix deadlock on import (#4693) (Joan Fontanals)

    🍹 Other Improvements

    • [a8329d66] - version: the next version will be 3.3.10 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.9(Apr 25, 2022)

    Release Note (3.3.9)

    Release time: 2022-04-25 19:59:19

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [162a59b6] - fix target executor in http (#4690) (Joan Fontanals)

    🍹 Other Improvements

    • [278a6992] - version: the next version will be 3.3.9 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.8(Apr 25, 2022)

    Release Note (3.3.8)

    Release time: 2022-04-25 17:42:17

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [81a0b8d5] - fix configuration of addresses in docker compose (#4684) (Joan Fontanals)

    🍹 Other Improvements

    • [e9a4d79c] - version: the next version will be 3.3.8 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.7(Apr 25, 2022)

    Release Note (3.3.7)

    Release time: 2022-04-25 14:16:33

    🙇 We'd like to thank all contributors for this new release! In particular, samsja, Tobias Jacobowitz, Johannes Messner, joschkabraun, Joan Fontanals, Jina Dev Bot, cristian, felix-wang, 🙇

    🆕 New Features

    • [6ac879e4] - add linkerd details in contributing (#4686) (samsja)
    • [59b76269] - add timeout_send (#4660) (Tobias Jacobowitz)
    • [4c4fc9e2] - monitoring: count the number of processed documents (#4672) (samsja)
    • [b3b139be] - pass environment to head and gateway (#4646) (Joan Fontanals)

    🐞 Bug fixes

    • [9b8deaa2] - dependency: prometheus client should be a core dependency (#4689) (samsja)
    • [96a83469] - better error messages when flow fails to start (#4652) (Johannes Messner)
    • [4168d44d] - set the multiprocessing start_method spawn (#4650) (felix-wang)

    📗 Documentation

    • [85b597ab] - fixed typos in limiting outstanding requests (#4611) (joschkabraun)
    • [b6aaa3ef] - add docs guide (#4659) (cristian)
    • [0c1468a7] - normalize usage of admonition boxes (#4662) (Johannes Messner)

    🍹 Other Improvements

    • [b086f0d9] - docs: update TOC (Jina Dev Bot)
    • [e7bd24ab] - version: the next version will be 3.3.7 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.6(Apr 22, 2022)

    Release Note (3.3.6)

    Release time: 2022-04-22 07:01:51

    🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, samsja, Jina Dev Bot, Johannes Messner, 🙇

    🐞 Bug fixes

    • [955a08d8] - fix circular import (#4649) (Joan Fontanals)
    • [af81dfe9] - reorder correctly the remark page (#4665) (samsja)

    📗 Documentation

    • [4a0a878e] - make all how-to titles sentence case (#4663) (Johannes Messner)

    🍹 Other Improvements

    • [0c8d6371] - docs: update TOC (Jina Dev Bot)
    • [d1add5a6] - version: the next version will be 3.3.6 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.5(Apr 21, 2022)

    Release Note (3.3.5)

    Release time: 2022-04-21 14:49:03

    🙇 We'd like to thank all contributors for this new release! In particular, Tobias Jacobowitz, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [8c53323e] - sandbox usage from k8s/docker-compose (#4671) (Tobias Jacobowitz)

    🍹 Other Improvements

    • [bf4c2be2] - version: the next version will be 3.3.5 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.3.4(Apr 21, 2022)

    Release Note (3.3.4)

    Release time: 2022-04-21 12:56:57

    🙇 We'd like to thank all contributors for this new release! In particular, cristian, Joan Fontanals, felix-wang, Tobias Jacobowitz, Johannes Messner, Jina Dev Bot, 🙇

    🐞 Bug fixes

    • [2c72e770] - pass extra search paths given from args (#4651) (felix-wang)
    • [8a3083f8] - relax time constraint in block exec test (#4653) (Tobias Jacobowitz)
    • [e92e26cc] - better reporting of errors in the head (#4640) (Johannes Messner)

    🧼 Code Refactoring

    • [b76d4774] - remove runtime backend cli arg (#4644) (Joan Fontanals)

    📗 Documentation

    • [d2161e66] - fix admonition in access flow page (#4655) (cristian)

    🍹 Other Improvements

    • [9140b134] - version: the next version will be 3.3.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
Connectionist Temporal Classification (CTC) decoding algorithms: best path, beam search, lexicon search, prefix search, and token passing. Implemented in Python.

CTC Decoding Algorithms Update 2021: installable Python package Python implementation of some common Connectionist Temporal Classification (CTC) decod

Harald Scheidl 690 May 11, 2022
txtai: Build AI-powered semantic search applications in Go

txtai: Build AI-powered semantic search applications in Go txtai executes machine-learning workflows to transform data and build AI-powered semantic s

NeuML 40 Apr 5, 2022
The following links explain a bit the idea of semantic search and how search mechanisms work by doing retrieve and rerank

Main Idea The following links explain a bit the idea of semantic search and how search mechanisms work by doing retrieve and rerank Semantic Search Re

Sergio Arnaud Gomez 2 Jan 28, 2022
Python library to make development of portfolio analysis faster and easier

Trafalgar Python library to make development of portfolio analysis faster and easier Installation ?? For the moment, Trafalgar is still in beta develo

Santosh Passoubady 517 May 20, 2022
This is the code for the EMNLP 2021 paper AEDA: An Easier Data Augmentation Technique for Text Classification

The baseline code is for EDA: Easy Data Augmentation techniques for boosting performance on text classification tasks

Akbar Karimi 66 May 27, 2022
:mag: Transformers at scale for question answering & neural search. Using NLP via a modular Retriever-Reader-Pipeline. Supporting DPR, Elasticsearch, HuggingFace's Modelhub...

Haystack is an end-to-end framework for Question Answering & Neural search that enables you to ... ... ask questions in natural language and find gran

deepset 4.7k May 26, 2022
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search

LightSpeech UnOfficial PyTorch implementation of LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search.

Rishikesh (ऋषिकेश) 50 May 10, 2022
code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling This repository contains PyTorch evaluation code, training code and pretrain

Facebook Research 84 May 20, 2022
Create a semantic search engine with a neural network (i.e. BERT) whose knowledge base can be updated

Create a semantic search engine with a neural network (i.e. BERT) whose knowledge base can be updated. This engine can later be used for downstream tasks in NLP such as Q&A, summarization, generation, and natural language understanding (NLU).

Diego 1 Mar 20, 2022
Explore different way to mix speech model(wav2vec2, hubert) and nlp model(BART,T5,GPT) together

SpeechMix Explore different way to mix speech model(wav2vec2, hubert) and nlp model(BART,T5,GPT) together. Introduction For the same input: from datas

Eric Lam 19 May 25, 2022
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 B) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 Billion Parameters) on a single 16 GB VRAM V100 Google Cloud instance with Huggingfa

null 245 May 8, 2022
Source code for the paper "TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations"

TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations Created by Jiahao Pang, Duanshun Li, and Dong Tian from InterDigital In

InterDigital 14 Apr 15, 2022
NeuTex: Neural Texture Mapping for Volumetric Neural Rendering

NeuTex: Neural Texture Mapping for Volumetric Neural Rendering Paper: https://arxiv.org/abs/2103.00762 Running Run on the provided DTU scene cd run ba

Fanbo Xiang 34 May 23, 2022
Build Text Rerankers with Deep Language Models

Reranker is a lightweight, effective and efficient package for training and deploying deep languge model reranker in information retrieval (IR), question answering (QA) and many other natural language processing (NLP) pipelines. The training procedure follows our ECIR paper Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline using a localized constrastive esimation (LCE) loss.

Luyu Gao 114 May 17, 2022
open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

中文开放信息抽取系统, open-information-extraction-system, build open-knowledge-graph(SPO, subject-predicate-object) by pyltp(version==3.4.0)

null 6 Sep 23, 2021
A tool helps build a talk preview image by combining the given background image and talk event description

talk-preview-img-builder A tool helps build a talk preview image by combining the given background image and talk event description Installation and U

PyCon Taiwan 4 May 14, 2022
Poetry PEP 517 Build Backend & Core Utilities

Poetry Core A PEP 517 build backend implementation developed for Poetry. This project is intended to be a light weight, fully compliant, self-containe

Poetry 232 May 15, 2022
Prithivida 450 May 26, 2022