An easier way to build neural search on the cloud

Overview

Jina banner

An easier way to build neural search on the cloud

Python 3.7 3.8 3.9 PyPI Docker Image Version (latest semver) CI CD codecov

Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g. text, images, video, audio) on the cloud.

โฑ๏ธ Time Saver - The design pattern of neural search systems, from zero to a production-ready system in minutes.

๐Ÿฑ Full-Stack Ownership - Keep an end-to-end stack ownership of your solution, avoid the integration pitfalls with fragmented, multi-vendor, generic legacy tools.

๐ŸŒŒ Universal Search - Large-scale indexing and querying of unstructured data: video, image, long/short text, music, source code, etc.

๐Ÿง  First-Class AI Models - First-class support for state-of-the-art AI models, easily usable and extendable with a Pythonic interface.

๐ŸŒฉ๏ธ Fast & Cloud Ready - Decentralized architecture from day one. Scalable & cloud-native by design: enjoy containerizing, distributing, sharding, async, REST/gRPC/WebSocket.

โค๏ธ Made with Love - Never compromise on quality, actively maintained by a passionate full-time, venture-backed team.


Docs โ€ข Hello World โ€ข Quick Start โ€ข Learn โ€ข Examples โ€ข Contribute โ€ข Jobs โ€ข Website โ€ข Slack

Installation

๐Ÿ“ฆ
x86/64,arm/v6,v7,v8 (Apple M1)
On Linux/macOS & Python 3.7/3.8/3.9 Docker Users
Standard pip install -U jina docker run jinaai/jina:latest
Daemon pip install -U "jina[daemon]" docker run --network=host jinaai/jina:latest-daemon
With Extras pip install -U "jina[devel]" docker run jinaai/jina:latest-devel
Dev/Pre-Release pip install --pre jina docker run jinaai/jina:master

Version identifiers are explained here. To install Jina with extra dependencies please refer to the docs. Jina can run on Windows Subsystem for Linux. We welcome the community to help us with native Windows support.

YAML Completion in IDE

Developing Jina app often means writing YAML configs. We provide a JSON Schema for your IDE to enable code completion, syntax validation, members listing and displaying help text. Here is a video tutorial to walk you through the setup.

PyCharm

  1. Click menu Preferences -> JSON Schema mappings;
  2. Add a new schema, in the Schema File or URL write https://api.jina.ai/schemas/latest.json; select JSON Schema Version 7;
  3. Add a file path pattern and link it to *.jaml and *.jina.yml.

VSCode

  1. Install the extension: YAML Language Support by Red Hat;
  2. In IDE-level settings.json add:
"yaml.schemas": {
    "https://api.jina.ai/schemas/latest.json": ["/*.jina.yml", "/*.jaml"],
}

Jina "Hello, World!" ๐Ÿ‘‹ ๐ŸŒ

Just starting out? Try Jina's "Hello, World" - jina hello --help

๐Ÿ‘— Fashion Image Search

A simple image neural search demo for Fashion-MNIST. No extra dependencies needed, simply run:

jina hello fashion  # more options in --help

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello fashion --workdir /j && open j/hello-world.html
# replace "open" with "xdg-open" on Linux
Click here to see console output

hello world console output

This downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute.

๐Ÿค– Covid-19 Chatbot

For NLP engineers, we provide a simple chatbot demo for answering Covid-19 questions. To run that:

pip install "jina[chatbot]"

jina hello chatbot

This downloads CovidQA dataset and tells Jina to index 418 question-answer pairs with DistilBERT. The index process takes about 1 minute on CPU. Then it opens a web page where you can input questions and ask Jina.



๐Ÿช† Multimodal Document Search

A multimodal-document contains multiple data types, e.g. a PDF document often contains figures and text. Jina lets you build a multimodal search solution in just minutes. To run our minimum multimodal document search demo:

pip install "jina[multimodal]"

jina hello multimodal

This downloads people image dataset and tells Jina to index 2,000 image-caption pairs with MobileNet and DistilBERT. The index process takes about 3 minute on CPU. Then it opens a web page where you can query multimodal documents. We have prepared a YouTube tutorial to walk you through this demo.




Get Started

๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฅš Fundamentals

CRUD Functions

First we look at basic CRUD operations. In Jina, CRUD corresponds to four functions: index (create), search (read), update, and delete. With Documents below as an example:

import numpy as np
from jina import Document
docs = [Document(id='๐Ÿฒ', embedding=np.array([0, 0]), tags={'guardian': 'Azure Dragon', 'position': 'East'}),
        Document(id='๐Ÿฆ', embedding=np.array([1, 0]), tags={'guardian': 'Vermilion Bird', 'position': 'South'}),
        Document(id='๐Ÿข', embedding=np.array([0, 1]), tags={'guardian': 'Black Tortoise', 'position': 'North'}),
        Document(id='๐Ÿฏ', embedding=np.array([1, 1]), tags={'guardian': 'White Tiger', 'position': 'West'})]

Let's build a Flow with a simple indexer:

from jina import Flow
f = Flow().add(uses='_index')

Document and Flow are basic concepts in Jina, which will be explained later. _index is a built-in embedding + structured storage that you can use out of the box.

Index
# save four docs (both embedding and structured info) into storage
with f:
    f.index(docs, on_done=print)
Search
# retrieve top-3 neighbours of ๐Ÿฒ, this print ๐Ÿฒ๐Ÿฆ๐Ÿข with score 0, 1, 1 respectively
with f:
    f.search(docs[0], top_k=3, on_done=lambda x: print(x.docs[0].matches))
{"id": "๐Ÿฒ", "tags": {"guardian": "Azure Dragon", "position": "East"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
{"id": "๐Ÿฆ", "tags": {"position": "South", "guardian": "Vermilion Bird"}, "embedding": {"dense": {"buffer": "AQAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
{"id": "๐Ÿข", "tags": {"guardian": "Black Tortoise", "position": "North"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAABAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
Update
# update ๐Ÿฒ embedding in the storage
docs[0].embedding = np.array([1, 1])
with f:
    f.update(docs[0])
Delete
# remove ๐Ÿฆ๐Ÿฒ Documents from the storage
with f:
    f.delete(['๐Ÿฆ', '๐Ÿฒ'])

For further details about CRUD functionality, checkout docs.jina.ai.

Document

Document is Jina's primitive data type. It can contain text, image, array, embedding, URI, and be accompanied by rich meta information. To construct a Document, you can use:

import numpy
from jina import Document

doc1 = Document(content=text_from_file, mime_type='text/x-python')  # a text document contains python code
doc2 = Document(content=numpy.random.random([10, 10]))  # a ndarray document

A Document can be recursed both vertically and horizontally to have nested Documents and matched Documents. To better see the Document's recursive structure, you can use .plot() function. If you are using JupyterLab/Notebook, all Document objects will be auto-rendered.

import numpy
from jina import Document

d0 = Document(id='๐Ÿฒ', embedding=np.array([0, 0]))
d1 = Document(id='๐Ÿฆ', embedding=np.array([1, 0]))
d2 = Document(id='๐Ÿข', embedding=np.array([0, 1]))
d3 = Document(id='๐Ÿฏ', embedding=np.array([1, 1]))

d0.chunks.append(d1)
d0.chunks[0].chunks.append(d2)
d0.matches.append(d3)

d0.plot()  # simply `d0` on JupyterLab
Click here to see more about MultimodalDocument

MultimodalDocument

A MultimodalDocument is a document composed of multiple Document from different modalities (e.g. text, image, audio).

Jina provides multiple ways to build a multimodal Document. For example, you can provide the modality names and the content in a dict:

from jina import MultimodalDocument
document = MultimodalDocument(modality_content_map={
    'title': 'my holiday picture',
    'description': 'the family having fun on the beach',
    'image': PIL.Image.open('path/to/image.jpg')
})

One can also compose a MultimodalDocument from multiple Document directly:

from jina.types import Document, MultimodalDocument

doc_title = Document(content='my holiday picture', modality='title')
doc_desc = Document(content='the family having fun on the beach', modality='description')
doc_img = Document(content=PIL.Image.open('path/to/image.jpg'), modality='image')
doc_img.tags['date'] = '10/08/2019'

document = MultimodalDocument(chunks=[doc_title, doc_description, doc_img])
Fusion Embeddings from Different Modalities

To extract fusion embeddings from different modalities Jina provides BaseMultiModalEncoder abstract class, which has a unique encode interface.

def encode(self, *data: 'numpy.ndarray', **kwargs) -> 'numpy.ndarray':
    ...

MultimodalDriver provides data to the MultimodalDocument in the correct expected order. In this example below, image embedding is passed to the encoder as the first argument, and text as the second.

jtype: MyMultimodalEncoder
with:
  positional_modality: ['image', 'text']
requests:
  on:
    [IndexRequest, SearchRequest]:
      - jtype: MultiModalDriver {}

Interested readers can refer to jina-ai/example: how to build a multimodal search engine for image retrieval using TIRG (Composing Text and Image for Image Retrieval) for the usage of MultimodalDriver and BaseMultiModalEncoder in practice.

Flow

Jina provides a high-level Flow API to simplify building CRUD workflows. To create a new Flow:

from jina import Flow
f = Flow().add()

This creates a simple Flow with one Pod. You can chain multiple .add()s in a single Flow.

To visualize the Flow, simply chain it with .plot('my-flow.svg'). If you are using a Jupyter notebook, the Flow object will be displayed inline without plot.

Gateway is the entrypoint of the Flow.

Get the vibe? Now we're talking! Let's learn more about the basic concepts and features of Jina:


๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฃ Basic

Feed Data

To use a Flow, open it via with context manager, like you would open a file in Python. Now let's create some empty Documents and index them:

from jina import Document

with Flow().add() as f:
    f.index((Document() for _ in range(10)))

Flow supports CRUD operations: index, search, update, delete. In addition, it also provides sugary syntax on ndarray, csv, ndjson and arbitrary files.

Input Example of index/search Explain
numpy.ndarray
with f:
  f.index_ndarray(numpy.random.random([4,2]))

Input four Documents, each document.blob is an ndarray([2])

CSV
with f, open('index.csv') as fp:
  f.index_csv(fp, field_resolver={'pic_url': 'uri'})

Each line in index.csv is constructed as a Document, CSV field pic_url mapped to document.uri.

JSON Lines/ndjson/LDJSON
with f, open('index.ndjson') as fp:
  f.index_ndjson(fp, field_resolver={'question_id': 'id'})

Each line in index.ndjson is constructed as a Document, JSON field question_id mapped to document.id.

Files with wildcards
with f:
  f.index_files(['/tmp/*.mp4', '/tmp/*.pdf'])

Each file captured is constructed as a Document, and Document content (text, blob, buffer) is auto-guessed & filled.

Fetch Result

Once a request is done, callback functions are fired. Jina Flow implements a Promise-like interface: You can add callback functions on_done, on_error, on_always to hook different events. In the example below, our Flow passes the message then prints the result when successful. If something goes wrong, it beeps. Finally, the result is written to output.txt.

def beep(*args):
    # make a beep sound
    import os
    os.system('echo -n "\a";')

with Flow().add() as f, open('output.txt', 'w') as fp:
    f.index(numpy.random.random([4, 5, 2]),
            on_done=print, on_error=beep, on_always=lambda x: fp.write(x.json()))

Add Logic

To add logic to the Flow, use the uses parameter to attach a Pod with an Executor. uses accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.

f = (Flow().add(uses=MyBertEncoder)  # the class of a Jina Executor
           .add(uses='docker://jinahub/pod.encoder.dummy_mwu_encoder:0.0.6-0.9.3')  # the image name
           .add(uses='myencoder.yml')  # YAML serialization of a Jina Executor
           .add(uses='!WaveletTransformer | {freq: 20}')  # inline YAML config
           .add(uses='_pass')  # built-in shortcut executor
           .add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}}))  # dict config object with __cls keyword

The power of Jina lies in its decentralized architecture: Each add creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.

Inter & Intra Parallelism

Chaining .add()s creates a sequential Flow. For parallelism, use the needs parameter:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', needs='gateway')
           .needs(['p1','p2', 'p3'], name='r1').plot())

p1, p2, p3 now subscribe to Gateway and conduct their work in parallel. The last .needs() blocks all Pods until they finish their work. Note: parallelism can also be performed inside a Pod using parallel:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', parallel=3)
           .needs(['p1','p3'], name='r1').plot())

Decentralized Flow

A Flow does not have to be local-only: You can put any Pod to remote(s). In the example below, with the host keyword gpu-pod, is put to a remote machine for parallelization, whereas other Pods stay local. Extra file dependencies that need to be uploaded are specified via the upload_files keyword.

123.456.78.9
# have docker installed
docker run --name=jinad --network=host -v /var/run/docker.sock:/var/run/docker.sock jinaai/jina:latest-daemon --port-expose 8000
# to stop it
docker rm -f jinad
Local
import numpy as np
from jina import Flow

f = (Flow()
     .add()
     .add(name='gpu_pod',
          uses='mwu_encoder.yml',
          host='123.456.78.9:8000',
          parallel=2,
          upload_files=['mwu_encoder.py'])
     .add())

with f:
    f.index_ndarray(np.random.random([10, 100]), output=print)

We provide a demo server on cloud.jina.ai:8000, give the following snippet a try!

from jina import Flow

with Flow().add().add(host='cloud.jina.ai:8000') as f:
    f.index(['hello', 'world'])

Asynchronous Flow

While synchronous from outside, Jina runs asynchronously under the hood: it manages the eventloop(s) for scheduling the jobs. If the user wants more control over the eventloop, then AsyncFlow can be used.

Unlike Flow, the CRUD of AsyncFlow accepts input and output functions as async generators. This is useful when your data sources involve other asynchronous libraries (e.g. motor for MongoDB):

from jina import AsyncFlow

async def input_function():
    for _ in range(10):
        yield Document()
        await asyncio.sleep(0.1)

with AsyncFlow().add() as f:
    async for resp in f.index(input_function):
        print(resp)

AsyncFlow is particularly useful when Jina is using as part of integration, where another heavy-lifting job is running concurrently:

async def run_async_flow_5s():  # WaitDriver pause 5s makes total roundtrip ~5s
    with AsyncFlow().add(uses='- !WaitDriver {}') as f:
        async for resp in f.index_ndarray(numpy.random.random([5, 4])):
            print(resp)

async def heavylifting():  # total roundtrip takes ~5s
    print('heavylifting other io-bound jobs, e.g. download, upload, file io')
    await asyncio.sleep(5)
    print('heavylifting done after 5s')

async def concurrent_main():  # about 5s; but some dispatch cost, can't be just 5s, usually at <7s
    await asyncio.gather(run_async_flow_5s(), heavylifting())

if __name__ == '__main__':
    asyncio.run(concurrent_main())

AsyncFlow is very useful when using Jina inside a Jupyter Notebook. As Jupyter/ipython already manages an eventloop and thanks to autoawait, AsyncFlow can run out-of-the-box in Jupyter.

That's all you need to know for understanding the magic behind hello-world. Now let's dive deeper into it!


๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฅ Breakdown of hello-world

Customize Encoder

Let's first build a naive image encoder that embeds images into vectors using an orthogonal projection. To do this, we simply inherit from BaseImageEncoder: a base class from the jina.executors.encoders module. We then override its __init__() and encode() methods.

import numpy as np
from jina.executors.encoders import BaseImageEncoder

class MyEncoder(BaseImageEncoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        np.random.seed(1337)
        H = np.random.rand(784, 64)
        u, s, vh = np.linalg.svd(H, full_matrices=False)
        self.oth_mat = u @ vh

    def encode(self, data: 'np.ndarray', *args, **kwargs):
        return (data.reshape([-1, 784]) / 255) @ self.oth_mat

Jina provides a family of Executor classes, which summarize frequently-used algorithmic components in neural search. This family consists of encoders, indexers, crafters, evaluators, and classifiers, each with a well-designed interface. You can find the list of all 107 built-in executors here. If they don't meet your needs, inheriting from one of them is the easiest way to bootstrap your own Executor. Simply use our Jina Hub CLI:

pip install jina[hub] && jina hub new

Test Encoder in Flow

Let's test our encoder in the Flow with some synthetic data:

def validate(req):
    assert len(req.docs) == 100
    assert NdArray(req.docs[0].embedding).value.shape == (64,)

f = Flow().add(uses='MyEncoder')

with f:
    f.index_ndarray(numpy.random.random([100, 28, 28]), on_done=validate)

All good! Now our validate function confirms that all one hundred 28x28 synthetic images have been embedded into 100x64 vectors.

Parallelism & Batching

By setting a larger input, you can play with batch_size and parallel:

f = Flow().add(uses='MyEncoder', parallel=10)

with f:
    f.index_ndarray(numpy.random.random([60000, 28, 28]), batch_size=1024)

Add Data Indexer

Now we need to add an indexer to store all the embeddings and the image for later retrieval. Jina provides a simple numpy-powered vector indexer NumpyIndexer, and a key-value indexer BinaryPbIndexer. We can combine them in a single YAML file:

jtype: CompoundIndexer
components:
  - jtype: NumpyIndexer
    with:
      index_filename: vec.gz
  - jtype: BinaryPbIndexer
    with:
      index_filename: chunk.gz
metas:
  workspace: ./
  • jtype: defines the class name of the structure;
  • with: defines arguments for initializing this class object.

๐Ÿ’ก Config your IDE to enable autocompletion on YAML

Essentially, the above YAML config is equivalent to the following Python code:

from jina.executors.indexers.vector import NumpyIndexer
from jina.executors.indexers.keyvalue import BinaryPbIndexer
from jina.executors.indexers import CompoundIndexer

a = NumpyIndexer(index_filename='vec.gz')
b = BinaryPbIndexer(index_filename='vec.gz')
c = CompoundIndexer()
c.components = lambda: [a, b]

Compose Flow from YAML

Now let's add our indexer YAML file to the Flow with .add(uses=). Let's also add two shards to the indexer to improve its scalability:

f = Flow().add(uses='MyEncoder', parallel=2).add(uses='myindexer.yml', shards=2).plot()

When you have many arguments, constructing a Flow in Python can get cumbersome. In that case, you can simply move all arguments into one flow.yml:

jtype: Flow
version: '1.0'
pods:
  - name: encode
    uses: MyEncoder
    parallel: 2
  - name:index
    uses: myindexer.yml
    shards: 2

And then load it in Python:

f = Flow.load_config('flow.yml')

Search

Querying a Flow is similar to what we did with indexing. Simply load the query Flow and switch from f.index to f.search. Say you want to retrieve the top 50 documents that are similar to your query and then plot them in HTML:

f = Flow.load_config('flows/query.yml')
with f:
    f.search_ndarray(numpy.random.random([10, 28, 28]), shuffle=True, on_done=plot_in_html, top_k=50)

Evaluation

To compute precision recall on the retrieved result, you can add _eval_pr, a built-in evaluator for computing precision & recall.

f = (Flow().add(...)
           .add(uses='_eval_pr'))

You can construct an iterator of query and groundtruth pairs and feed to the flow f, via:

from jina import Document

def query_generator():
    for _ in range(10):
        q = Document()
        # now construct expect matches as groundtruth
        gt = Document(q, copy=True)  # make sure 'gt' is identical to 'q'
        gt.matches.append(...)
        yield q, gt

f.search(query_iterator, ...)

REST Interface

In practice, the query Flow and the client (i.e. data sender) are often physically separated. Moreover, the client may prefer to use a REST API rather than gRPC when querying. You can set port_expose to a public port and turn on REST support with restful=True:

f = Flow(port_expose=45678, restful=True)

with f:
    f.block()

That is the essence behind jina hello fashion. It is merely a taste of what Jina can do. Weโ€™re really excited to see what you do with Jina! You can easily create a Jina project from templates with one terminal command:

pip install jina[hub] && jina hub new --type app

This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Learn

Examples (View all)

Example code to build your own projects

๐Ÿ“„

NLP Semantic Wikipedia Search with Transformers and DistilBERT

Brand new to neural search? See a simple text-search example to understand how Jina works

๐Ÿ“„

Add Incremental Indexing to Wikipedia Search

Index more effectively by adding incremental indexing to your Wikipedia search

๐Ÿ“„

Search Lyrics with Transformers and PyTorch

Get a better understanding of chunks by searching a lyrics database. Now with shiny front-end!

๐Ÿ–ผ๏ธ

Google's Big Transfer Model in (Pokรฉ-)Production

Use SOTA visual representation for searching Pokรฉmon!

๐ŸŽง

Search YouTube audio data with Vggish

A demo of neural search for audio data based Vggish model.

๐ŸŽž๏ธ

Search Tumblr GIFs with KerasEncoder

Use prefetching and sharding to improve the performance of your index and query Flow when searching animated GIFs.

Please check our examples repo for advanced and community-submitted examples.

Want to read more? Check our Founder Han Xiao's blog and our official blog.

Documentation

Apart from the learning resources above, We highly recommended you go through our documentation to master Jina.

Our docs are built on every push, merge, and release of Jina's master branch. Documentation for older versions is archived here.

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors โœจ

All Contributors

Community

  • Code of conduct - play nicely with the Jina community
  • Slack workspace - join #general on our Slack to meet the team and ask questions
  • YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow and interact with us using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020-2021 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Comments
  • Text-by-Audio

    Text-by-Audio

    Hi,

    I want to retrieve text by searching for an audio using AudioClip model.

    First, I created indexing of text (car-horn, coughing, alarm-clock, thunderstorm etc) and using AudioClipTextEncoder for embedding.

    After that I am searching for an audio where i am using AudioClipEncoder for embedding.

    For both text and audio indexing simple-indexer is used.

    While searching for an audio i created chunks using segmenter and using MyRanker for ranking scores where i modified some script.

    The output is: input_audio: AudioCLIP/demo/audio/thunder_3-144891-B-19.wav, len-of-chunks:1 +- id: e5eceb26737311ec83fd2fdc0fc83b8e score: 0.931110, cat input_audio: AudioCLIP/demo/audio/coughing_1-58792-A-24.wav, len-of-chunks:1 +- id: e5ecf968737311ec83fd2fdc0fc83b8e score: 0.923869, cat input_audio: AudioCLIP/demo/audio/cat_3-95694-A-5.wav, len-of-chunks:1 +- id: e5ed00de737311ec83fd2fdc0fc83b8e score: 0.845475, cat input_audio: AudioCLIP/demo/audio/car_horn_1-24074-A-43.wav, len-of-chunks:1 +- id: e5ed0778737311ec83fd2fdc0fc83b8e score: 0.928283, cat input_audio: AudioCLIP/demo/audio/alarm_clock_3-120526-B-37.wav, len-of-chunks:1 +- id: e5ed0dcc737311ec83fd2fdc0fc83b8e score: 0.919222, alarm_clock

    As above for 5 inputs only 2 outputs are correct. Please let me know why is that so ? Since AudioClipEncoder is giving correct outputs for all audios.

    opened by snaaz21 55
  • [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    Describe your proposal/problem

    Dear Jina Team,

    Recently, I tried nlp-simple example and replaced TransformerTorchEncoder with TextPaddlehubEncoder. However, I got the following error.

    Traceback (most recent call last):
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 73, in _load_executor
        self._executor = BaseExecutor.load_config(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 531, in load_config
        return JAML.load(tag_yml, substitute=False)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 89, in load
        r = yaml.load(stream, Loader=JinaLoader)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
        return loader.get_single_data()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
        return self.construct_document(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
        data = self.construct_object(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
        data = constructor(self, node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 422, in _from_yaml
        return get_parser(cls, version=data.get('version', None)).parse(cls, data)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/parsers/executor/legacy.py", line 130, in parse
        obj = cls(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 82, in __call__
        getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 174, in _post_init_wrapper
        self.post_init()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/hub/encoders/nlp/TextPaddlehubEncoder/__init__.py", line 38, in post_init
        self.model = hub.Module(name=self.model_name)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 171, in __new__
        module = cls.init_with_name(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 263, in init_with_name
        user_module_cls = manager.install(name=name, version=version, source=source, update=update, branch=branch)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 188, in install
        return self._install_from_name(name, version)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 263, in _install_from_name
        return self._install_from_url(item['url'])
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 256, in _install_from_url
        return self._install_from_archive(file)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 361, in _install_from_archive
        return self._install_from_directory(directory)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 345, in _install_from_directory
        hub_module_cls = HubModule.load(self._get_normalized_path(module_info.name))
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 219, in load
        paddle.set_device(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/device.py", line 166, in set_device
        framework._set_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 317, in _set_expected_place
        _set_dygraph_tracer_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 311, in _set_dygraph_tracer_expected_place
        _dygraph_tracer_._expected_place = place
    OSError: (External)  Cuda error(3), initialization error.
      [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:229)
    

    It seems to be related with https://github.com/PaddlePaddle/Paddle/issues/25185.

    However, I have no idea how to fix this. Could you help me to figure it out? Thank you very much!


    Environment

    paddlehub==2.0.0 paddlenlp==2.0.1 paddlepaddle-gpu==2.0.2.post90 jina==1.3.0

    Screenshots

    image

    area/core type/bug 
    opened by ryh95 47
  • feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    Goals:

    • resolves #5155
    • [x] Integrate OpenTelemetry API and SDK.
    • ~[ ] Provide environment variable configurations to enable tracking when required. Use console exporter for now.~
    • [x] Trace gRPC requests within the Flow.
    • [x] Add helpers for creating traces on request methods with default span attributes.
    • ~[ ] Convert send_health_check_sync or is_ready method to async to prevent the grpc aio interceptor from throwing and capturing an exception.~
    • [x] Extract tracing context from the server and make it available for the Executor methods in the kwargs list or arguments.
    • [ ] check and update documentation. See guide and ask the team.

    Sample Usage

    Flow

    jtype: Flow
    version: '1'
    with:
      protocol: grpc
      port: 54321
      tracing: true
      traces_exporter_host: '0.0.0.0'
      traces_exporter_port: 4317
    executors:
      - uses: executor1/config.yml
        name: toyExecutor
      - uses: executor2/config.yml
        name: toyExecutor2
    

    Executor

    import functools
    
    from jina import DocumentArray, Executor, requests
    from opentelemetry.context.context import Context
    from opentelemetry.semconv.trace import SpanAttributes
    from opentelemetry.trace import Status, StatusCode
    
    
    class MyExecutor(Executor):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
        @requests
        def foo(self, docs: DocumentArray, tracing_context: Context, **kwargs):
            with self.tracer.start_span("foo", context=tracing_context) as span:
                try:
                    span.set_attribute("len_added_docs", len(docs))
                    span.set_attribute(SpanAttributes.RPC_METHOD, functools.__name__)
    
                    docs[0].text = 'hello, world!'
                    docs[1].text = 'goodbye, world!'
                except Exception as ex:
                    span.set_status(Status(StatusCode.ERROR))
                    span.record_exception(ex)
                finally:
                    span.set_status(Status(StatusCode.OK))
    

    Client

    from jina import Client, DocumentArray
    import time
    
    if __name__ == '__main__':
        c = Client(
            host='grpc://0.0.0.0:54321',
            tracing=True,
            traces_exporter_host='0.0.0.0',
            traces_exporter_port=4317,
        )
    
        da = c.post('/', DocumentArray.empty(4))
        print(da.texts)
    
        time.sleep(3)
    
    

    Collecting Data

    Please check the docker-compose.yml and otel-collector-config.yml under the folder tests/integration/instrumentation for running the OpenTelemetry collector and Jaeger UI locally.

    size/L area/core size/S area/testing size/XL area/docs component/resource area/setup component/client area/cli 
    opened by girishc13 45
  • No documents are found with BinaryPbIndexer in 1.1.2

    No documents are found with BinaryPbIndexer in 1.1.2

    Versions 1.0.11: everything works as expected 1.1.2: BinaryPbIndexer doesn't find any documents

    Detailed description He Jina team, when I upgrade to 1.1.2 and reindex my dataset, the BinaryPbIndexer no longer works. It doesn't raise any errors, but I just can't find any documents. Just as a remark, I use custom ids which always consist of 33 characters. Where there any changes for the BinaryPbIndexer, the KVIndexDriver or the KVSearchDriver?

    area/core priority/critical type/bug status/done 
    opened by janandreschweiger 39
  • Permanently delete documents from storage

    Permanently delete documents from storage

    Hey Jina team!

    As described here documents persist the deletion and are only marked as inaccessible.

    My question: Is there a possibility to delete documents from storage without reindexing everything again? Deletion performance isn't an issue for us, but reindexing everything is very inconvenient in our use case.

    Thanks! I really appreciate your kind support.

    Sub-tasks

    • [x] Delete from NpIndexer. @ #2046
    • [x] Delete from BinaryPbIndexer. @ https://github.com/jina-ai/jina/pull/2102
    area/core priority/important-soon type/feature-request status/done 
    opened by ace-kay-law-neo 39
  • File seems to be closed when trying to index

    File seems to be closed when trying to index

    Yes @JoanFM, using the prefetch argument in the flow-config solved the original problem. I tried to pass the batch_size and the request_size like you, but I still get the same error. I have some additional logs, which might indicate what causes the problem. As I said, the first few batches are indexed successfully, but at some point I get this error:

            encoder@63[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            encoder@63[I]:#sent: 3 #recv: 4 sent_size: 13.8 MB recv_size: 1.2 MB
          chunk_idx@66[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            encoder@63[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
       NumpyIndexer@66[S]:artifacts of this executor (wrap_idx) is persisted to /var/huforce/fs/workspace/wrap_idx-0/wrap_idx.bin
    BinaryPbIndexer@66[S]:artifacts of this executor (chunk_idx) is persisted to /var/huforce/fs/workspace/chunk_idx-0/chunk_idx.bin
          chunk_idx@66[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
          chunk_idx@66[I]:#sent: 3 #recv: 4 sent_size: 1.9 MB recv_size: 15.6 MB
            doc_idx@67[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธdoc_idx/ZEDRuntimeโ–ธโš
    DEBUG | Message: Successfully indexed 4 documents!
            doc_idx@67[S]:artifacts of this executor (doc_idx_file) is persisted to /var/huforce/fs/workspace/doc_idx_file-0/doc_idx_file.bin
            doc_idx@67[I]:#sent: 3 #recv: 4 sent_size: 6.3 KB recv_size: 2.2 MB
           | ๐Ÿ“ƒ     16 โฑ๏ธ 262.9s ๐ŸŽ 0.1/s      4   requests        gateway@72[I]:send: 5 recv: 4 pending: 1
            crafter@60[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธโš
    DEBUG | Message: File-Info: research_admission.txt
    DEBUG | Message: File-Info: cloudcomputing.pptx
    DEBUG | Message: File-Info: WS_2012_White_Paper_Hyper-V.pdf
    DEBUG | Message: File-Info: writing-sql-queries.docx
            crafter@60[I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            crafter@60[I]:#sent: 5 #recv: 6 sent_size: 1.2 MB recv_size: 5.2 KB
            encoder@63[I]:#sent: 4 #recv: 5 sent_size: 15.6 MB recv_size: 1.2 MB
          chunk_idx@66[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            encoder@63[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
          chunk_idx@66[I]:#sent: 4 #recv: 5 sent_size: 2.2 MB recv_size: 16.1 MB
    DEBUG | Message: Encoding 552 paragraphs
          chunk_idx@66[E]:ValueError('write to closed file')
     add "--quiet-error" to suppress the exception details
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 198, in _msg_callback
        self._zmqlet.send_message(self._callback(msg))
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 184, in _callback
        self._pre_hook(msg)._handle(msg)._post_hook(msg)
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 177, in _handle
        self._executor(self.request_type)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/__init__.py", line 595, in __call__
        d()
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/__init__.py", line 381, in __call__
        self._apply_all(documents, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/index.py", line 58, in _apply_all
        self.exec_fn(keys, values)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/decorators.py", line 44, in arg_wrapper
        f = func(self, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/indexers/keyvalue.py", line 178, in add
        self.write_handler.header.write(
    ValueError: write to closed file
            doc_idx@67[I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโœ–โ–ธdoc_idx/ZEDRuntimeโ–ธโš
    dev_ai            | DEBUG | Message: Successfully indexed 4 documents!
    

    What's also strange is, that the failed documents for the chunk-idx are passed on to the doc-idx, as you can see in the logs.

    My flow for indexing:

    !AsyncFlow
    with:
      logserver: false
      show-exc-info: true
      prefetch: 2
      # request-size: 2
      # batch_size: 2
    pods:
      crafter:
        uses: pods/craft.yml
      encoder:
        uses: pods/encode.yml
        timeout_ready: 600000
      chunk_idx:
        uses: pods/chunk.yml
      doc_idx:
        uses: pods/doc.yml
    

    Originally posted by @nero-nazok in https://github.com/jina-ai/jina/issues/2298#issuecomment-817084516

    area/core type/bug 
    opened by JoanFM 32
  • DOCQA..

    DOCQA..

    I am trying to parse a pdf & apply the DOCQA model to it.Roberta as well as BERT..!! But I am not getting right answer for it.. whereas independently it's works accurately... !!

    This is my flow.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
    

    This is my query.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
      - name: generator
        uses: Generator
        py_modules: "flows/generator_roberta.py"
    
    opened by jyotikhetan 29
  • feat: allow multiple port and protocols for gateway

    feat: allow multiple port and protocols for gateway

    This PR adds support for multiple ports and protocols for the gateway runtime.

    • [x] Gateway parser can receive list or single value
    • [x] Flow __init__ and config_gateway can receive multiple or single value. Flow.port and Flow.protocol return single or multiple values
    • [x] GatewayRuntime expects list of values
    • [x] BaseGateway can receive runtime args
    • [ ] (optional) move injected dependencies to runtime_args, TBD in https://github.com/jina-ai/jina/issues/5396
    • [x] implemented Gateway classes uses the port and protocol from runtime args
    • [x] support in JAML
    • [x] k8s gateway service
    • [x] docker compose gateway service
    • [x] health checks
    • NetworkChecker
    • docker and k8s checks
    • Flow pod checks
    • [ ] ~~apply the same principle for port_monitoring~~ TBD in a separate PR: https://github.com/jina-ai/jina/issues/5352
    • [x] fix gateway output: https://github.com/jina-ai/jina/issues/5345
    • [ ] pass CI
    • [x] Add tests follow up: https://github.com/jina-ai/jina/issues/5393 closes: https://github.com/jina-ai/jina/issues/5345, https://github.com/jina-ai/jina/issues/5343
    size/M size/L area/core area/helper area/testing size/XL area/docs component/resource area/cli component/jaml 
    opened by alaeddine-13 28
  • Gateway error: received an empty stream from the client

    Gateway error: received an empty stream from the client

    Description

    I encountered an error in flow.post method while trying to run python app.py -t index in multires-lyrics-search after cloning this repository: examples. The error message was: gateway@61291[E]:receive an empty stream from the client! please check your client's inputs, you can use "Client.check_input(inputs)".


    Environment

    • jina 2.0.18
    • jina-proto 0.0.85
    • jina-vcs-tag (unset)
    • libzmq 4.3.4
    • pyzmq 1.21.5
    • protobuf 3.20.0
    • proto-backend cpp
    • grpcio 1.44.0
    • pyyaml 6.0
    • python 3.7.13
    • platform Linux
    • platform-release 5.10.105-1-MANJARO
    • platform-version #1 SMP PREEMPT Fri Mar 11 14:12:33 UTC 2022
    • architecture x86_64
    • processor
    • uid 5025763201072
    • session-id 7d472a12-b722-11ec-a806-049226d49030
    • uptime 2022-04-08T15:58:49.863145
    • ci-vendor (unset)
    • JINA_ARRAY_QUANT (unset)
    • JINA_CONTROL_PORT (unset)
    • JINA_DEFAULT_HOST (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_HUBBLE_REGISTRY (unset)
    • JINA_HUB_CACHE_DIR (unset)
    • JINA_HUB_ROOT (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_ID (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_LOG_WORKSPACE (unset)
    • JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
    • JINA_POD_NAME (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_VCS_VERSION (unset)
    • JINA_MP_START_METHOD (unset)
    opened by Jackal1586 28
  • feat: add grpc health checking

    feat: add grpc health checking

    This PR aims to change the way health check is done in the core for Executors but also for any runtime using grpc. We aim to use the standard grpc https://github.com/grpc/grpc/blob/master/doc/health-checking.md as suggested by grpc, and which is something that lots of Load Balancers, etc ... will use to make sure grpc-based microservices are healthy.

    This also has the benefit that ControlRequest can be removed from the code.

    area/core area/helper area/testing area/network size/XL area/docs component/resource area/setup component/client component/proto area/entrypoint component/type 
    opened by JoanFM 27
  • feat: remove head for non sharded deployments

    feat: remove head for non sharded deployments

    Remove heads for non sharded Deployments

    Description

    This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers. Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

    This is a breaking change :boom: It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

    Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

    • [x] Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD
    • [x] Set all worker Pods in the Gateway (Outside K8s)
    • [x] Make the gateway send to worker Pods
    • [x] Dont create heads for Deployments with no shards
    • [x] Implement reducing after multiple needs in the Gateway
    • [x] Adapt tests
    • [x] Change docs

    Closes #4503 (issue)

    area/core area/helper area/testing size/XL area/docs area/cli area/daemon 
    opened by jacobowitz 27
  • Refactor: use K8sDeploymentConfig inside Deployment to generate kubernetes yamls

    Refactor: use K8sDeploymentConfig inside Deployment to generate kubernetes yamls

    After Deployment is exposed to serve Executors, we want to use it in order to generate kubernetes yaml as well. This means, the Deployment should use K8sDeploymentConfig to implement to_kubernetes_yaml.

    Potentially, we also want to refactor Flow.to_kubernetes_yaml to use the new method and maybe the same for Executor (or deprecate Executor.to_kubernetes_yaml)

    opened by alaeddine-13 0
  • fix(hub-integration): apply changes needed for Executor arm64 support

    fix(hub-integration): apply changes needed for Executor arm64 support

    Goals:

    • jina-hubble-sdk recently provided --platform parameter for Executor push and --prefer-platform for Executor pull
    • Update related tests to new version of fetch_meta provided by the SDK
    • Automated updates of command-line arguments
    • [ ] check and update documentation. See guide and ask the team.
    area/core size/S area/testing area/docs size/XS area/cli 
    opened by nomagick 2
  • Gateway down when doing simple PDF extraction

    Gateway down when doing simple PDF extraction

    Describe the bug Using the PDFSegmenter, a gateway runtime error occurs. I can confirm that the issue is not with the text or image extraction of the PDFSegmenter as that runs without error. After the code runs, it seems to hang for about 10 additional seconds then I get an error message:

    ERROR  gateway/rep-0/GatewayRuntime@29851 Error while getting responses from deployments: failed to connect to all addresses |Gateway: Communication error with deployment segmenter at         [01/05/23 23:57:41]
           address(es) {'0.0.0.0:59488'}. Head or worker(s) may be down.                                                                                                                                               
    ERROR  gateway/rep-0/GatewayRuntime@29851 Error while getting responses from deployments: failed to connect to all addresses |Gateway: Communication error with deployment segmenter at                            
           address(es) {'0.0.0.0:59488'}. Head or worker(s) may be down. 
    

    Additional details:

    test.py:

    from docarray import DocumentArray
    from jina import Flow
    from pdf_segmenter import PDFSegmenter
    
    # print-to-pdf of https://courses.cs.vt.edu/csonline/AI/Lessons/VisualProcessing/OCRscans_files/bowers.jpg
    docs = DocumentArray.from_files(["data/*.pdf"],)
    
    print(docs.summary())
    
    print(docs[0].summary())
    
    for doc in docs:
      doc.load_uri_to_blob()
    
    flow = Flow(protocol='http',timeout_ctrl=100000,timeout_ready=100000,timeout_send=100000, prefetch=1)\
        .add(uses=PDFSegmenter, name="segmenter", install_requirements=True,
    )
    
    with flow:
        output = flow.index(docs, return_results=True)
    
    
    # print(output.summary())
    
    # for o in output:
    #     print('-----------------')
    #     print(o.summary())
    #     print(len(o.chunks))
    

    pdf_segmenter.py:

    import io
    from typing import List
    
    import fitz
    import numpy as np
    import pdfplumber
    from jina import Document, DocumentArray, Executor, requests
    from jina.logging.logger import JinaLogger
    
    
    class PDFSegmenter(Executor):
        def __init__(
            self,
            *args,
            **kwargs,
        ):
            """
            :class:`PDFSegmenter` Extracts data (text and images) from PDF files.
            Stores images (`mime_type`=image/*) on chunk level ('c') and text segments (`mime_type`=text/plain)
            on chunk level ('c') in the root ('r') Document.
            """
            super().__init__(*args, **kwargs)
            self.logger = JinaLogger(context=self.__class__.__name__)
    
        @requests
        def craft(self, docs: DocumentArray, **kwargs):
            """
            Read PDF files. Extracts data from them.
            Checks if the input is a string of the filename,
            or if it's the file in bytes.
            It will then extract the data from the file, creating a list for images,
            and text.
            :param docs: Array of Documents.
            """
            for doc in docs:
                pdf_img, pdf_text = self._parse_pdf(doc)
                # print('getting images....')
                if pdf_img is not None:
                    images = self._extract_image(pdf_img)
                    doc.chunks.extend(
                        [Document(tensor=img, mime_type='image/*') for img in images]
                    )
                # print('getting text....')
                if pdf_text is not None:
                    texts = self._extract_text(pdf_text)
                    # print('texts: ', texts)
                    # for t in texts:
                    #     print(t)
                    doc.chunks.extend(
                        [Document(text=t, mime_type='text/plain') for t in texts]
                    )
                print('doc: ', doc)
            print('should be done')
            return
    
        def _parse_pdf(self, doc: Document):
            pdf_img = None
            pdf_text = None
            try:
                # when loading from URI, we should prioritize blob
                # order is important. check test `tests/unit/test_exec.py::test_order_blob_uri`
                if doc.blob:
                    pdf_img = fitz.open(stream=doc.blob, filetype='pdf')
                    pdf_text = pdfplumber.open(io.BytesIO(doc.blob))
                elif doc.uri:
                    pdf_img = fitz.open(doc.uri)
                    pdf_text = pdfplumber.open(doc.uri)
            except Exception as ex:
                self.logger.error(f'Failed to open due to: {ex}')
            return pdf_img, pdf_text
    
        def _extract_text(self, pdf_text) -> List[str]:
            # Extract text
            with pdf_text:
                texts = []
                count = len(pdf_text.pages)
                for i in range(count):
                    page = pdf_text.pages[i]
                    texts.append(page.extract_text(x_tolerance=1, y_tolerance=1))
                return texts
    
        def _extract_image(self, pdf_img) -> List['np.ndarray']:
            with pdf_img:
                images = []
                for page in range(len(pdf_img)):
                    print('current images: ', images)
                    for img in pdf_img.get_page_images(page):
                        xref = img[0]
                        pix = fitz.Pixmap(pdf_img, xref)
                        # read data from buffer and reshape the array into 3-d format
                        np_arr = (
                            np.frombuffer(pix.samples, dtype=np.uint8)
                            .reshape(pix.h, pix.w, pix.n)
                            .astype('float32')
                        )
                        if pix.n - pix.alpha < 4:  # if gray or RGB
                            if pix.n == 1:  # convert gray to rgb
                                images.append(np.concatenate((np_arr,) * 3, -1))
                            elif pix.n == 4:  # remove transparency layer
                                images.append(np_arr[..., :3])
                            else:
                                images.append(np_arr)
                        else:  # if CMYK:
                            pix = fitz.Pixmap(fitz.csRGB, pix)  # Convert to RGB
                            np_arr = (
                                np.frombuffer(pix.samples, dtype=np.uint8)
                                .reshape(pix.h, pix.w, pix.n)
                                .astype('float32')
                            )
                            images.append(np_arr)
            return images
    

    Here is the output with running JINA_LOG_LEVEL=DEBUG python test.py:

    โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Documents Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
    โ”‚                                                       โ”‚
    โ”‚   Type                   DocumentArrayInMemory        โ”‚
    โ”‚   Length                 1                            โ”‚
    โ”‚   Homogenous Documents   True                         โ”‚
    โ”‚   Common Attributes      ('id', 'mime_type', 'uri')   โ”‚
    โ”‚   Multimodal dataclass   False                        โ”‚
    โ”‚                                                       โ”‚
    โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
    โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Attributes Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
    โ”‚                                                              โ”‚
    โ”‚   Attribute   Data type   #Unique values   Has empty value   โ”‚
    โ”‚  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”‚
    โ”‚   id          ('str',)    1                False             โ”‚
    โ”‚   mime_type   ('str',)    1                False             โ”‚
    โ”‚   uri         ('str',)    1                False             โ”‚
    โ”‚                                                              โ”‚
    โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
    None
    ๐Ÿ“„ Document: b791272a4c237eacf92d9538b50405e8
    โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
    โ”‚ Attribute               โ”‚ Value                                              โ”‚
    โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
    โ”‚ mime_type               โ”‚ application/pdf                                    โ”‚
    โ”‚ uri                     โ”‚ data/rg.25si055505.pdf                             โ”‚
    โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
    None
    WARNIโ€ฆ JINA@42018 Error getting the directory name from PDFSegmenter. `--install-requirements` option is only valid when `uses` is a configuration file.                                        [01/06/23 00:12:18]
    โ ‹ Waiting ... โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 0/0 -:--:--DEBUG  segmenter/rep-0@42052 <pdf_segmenter.PDFSegmenter object at 0x7f950401ac40> is successfully loaded!                                                                                      [01/06/23 00:12:18]
    DEBUG  segmenter/rep-0@42052 start listening on 0.0.0.0:53273                                                                                                                                                      
    DEBUG  segmenter/rep-0@42052 run grpc server forever                                                                                                                                                               
    DEBUG  gateway/rep-0/GatewayRuntime@42053 adding connection for deployment segmenter/heads/0 to grpc://0.0.0.0:53273                                                                            [01/06/23 00:12:18]
    DEBUG  gateway/rep-0/GatewayRuntime@42053 start server bound to 0.0.0.0:56301                                                                                                                                      
    DEBUG  segmenter/rep-0@42018 ready and listening                                                                                                                                                [01/06/23 00:12:18]
    DEBUG  gateway/rep-0@42018 ready and listening                                                                                                                                                  [01/06/23 00:12:18]
    โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ ๐ŸŽ‰ Flow is ready to serve! โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ ๐Ÿ”— Endpoint โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
    โ”‚  โ›“      Protocol                   GRPC  โ”‚
    โ”‚  ๐Ÿ         Local          0.0.0.0:56301  โ”‚
    โ”‚  ๐Ÿ”’      Private     10.120.14.79:56301  โ”‚
    โ”‚  ๐ŸŒ       Public    165.124.167.4:56301  โ”‚
    โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
    DEBUG  Flow@42018 2 Deployments (i.e. 2 Pods) are running in this Flow                                                                                                                          [01/06/23 00:12:19]
    DEBUG  GRPCClient@42018 connected to 0.0.0.0:56301                                                                                                                                              [01/06/23 00:12:20]
    DEBUG  segmenter/rep-0@42052 got an endpoint discovery request                                                                                                                                  [01/06/23 00:12:20]
    DEBUG  segmenter/rep-0@42052 recv DataRequest at /index with id: c6f4046393a447f4b794046211a3b2cb                                                                                                                  
    doc:  <Document ('id', 'blob', 'mime_type', 'uri', 'chunks') at b791272a4c237eacf92d9538b50405e8>
    should be done
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to segmenter errored, with error <AioRpcError of RPC that terminated with:                                                                  [01/06/23 00:12:39]
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "Socket closed"                                                                                                                                                                           
                   debug_error_string = "{"created":"@1672985559.056961950","description":"Error received from peer                                                                                                    
           ipv4:0.0.0.0:53273","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Socket closed","grpc_status":14}"                                                                                 
           > and for the 1th time.                                                                                                                                                                                     
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to deployment segmenter failed with error <AioRpcError of RPC that terminated with:                                                                            
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "Socket closed"                                                                                                                                                                           
                   debug_error_string = "{"created":"@1672985559.056961950","description":"Error received from peer                                                                                                    
           ipv4:0.0.0.0:53273","file":"src/core/lib/surface/call.cc","file_line":966,"grpc_message":"Socket closed","grpc_status":14}"                                                                                 
           >, for retry attempt 1/3. Trying next replica, if available.                                                                                                                                                
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to segmenter errored, with error <AioRpcError of RPC that terminated with:                                                                                     
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "failed to connect to all addresses"                                                                                                                                                      
                   debug_error_string = "{"created":"@1672985559.072485749","description":"Failed to pick                                                                                                              
           subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1672985559.072484937","description":"failed to connect to                     
           all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"                                                                                                          
           > and for the 2th time.                                                                                                                                                                                     
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to deployment segmenter failed with error <AioRpcError of RPC that terminated with:                                                                            
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "failed to connect to all addresses"                                                                                                                                                      
                   debug_error_string = "{"created":"@1672985559.072485749","description":"Failed to pick                                                                                                              
           subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1672985559.072484937","description":"failed to connect to                     
           all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"                                                                                                          
           >, for retry attempt 2/3. Trying next replica, if available.                                                                                                                                                
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to segmenter errored, with error <AioRpcError of RPC that terminated with:                                                                                     
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "failed to connect to all addresses"                                                                                                                                                      
                   debug_error_string = "{"created":"@1672985559.083677854","description":"Failed to pick                                                                                                              
           subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1672985559.083662525","description":"failed to connect to                     
           all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"                                                                                                          
           > and for the 3th time.                                                                                                                                                                                     
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to deployment segmenter failed with error <AioRpcError of RPC that terminated with:                                                                            
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "failed to connect to all addresses"                                                                                                                                                      
                   debug_error_string = "{"created":"@1672985559.083677854","description":"Failed to pick                                                                                                              
           subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1672985559.083662525","description":"failed to connect to                     
           all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"                                                                                                          
           >, for retry attempt 3/3. Trying next replica, if available.                                                                                                                                                
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call to segmenter errored, with error <AioRpcError of RPC that terminated with:                                                                                     
                   status = StatusCode.UNAVAILABLE                                                                                                                                                                     
                   details = "failed to connect to all addresses"                                                                                                                                                      
                   debug_error_string = "{"created":"@1672985559.095512863","description":"Failed to pick                                                                                                              
           subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3260,"referenced_errors":[{"created":"@1672985559.095511871","description":"failed to connect to                     
           all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":167,"grpc_status":14}]}"                                                                                                          
           > and for the 4th time.                                                                                                                                                                                     
    DEBUG  gateway/rep-0/GatewayRuntime@42053 GRPC call for segmenter failed, retries exhausted                                                                                                                        
    DEBUG  gateway/rep-0/GatewayRuntime@42053 resetting connection for segmenter to 0.0.0.0:53273                                                                                                                      
    ERROR  gateway/rep-0/GatewayRuntime@42053 Error while getting responses from deployments: failed to connect to all addresses |Gateway: Communication error with deployment segmenter at                            
           address(es) {'0.0.0.0:53273'}. Head or worker(s) may be down.                                                                                                                                               
    ERROR  GRPCClient@42018 gRPC error: StatusCode.UNAVAILABLE failed to connect to all addresses |Gateway: Communication error with deployment segmenter at address(es) {'0.0.0.0:53273'}. Head or [01/06/23 00:12:39]
           worker(s) may be down.                                                                                                                                                                                      
           The ongoing request is terminated as the server is not available or closed already.                                                                                                                         
    DEBUG  gateway/rep-0@42018 waiting for ready or shutdown signal from runtime                                                                                                                    [01/06/23 00:12:39]
    DEBUG  gateway/rep-0@42018 terminate                                                                                                                                                                               
    DEBUG  gateway/rep-0@42018 terminating the runtime process                                                                                                                                                         
    DEBUG  gateway/rep-0@42018 runtime process properly terminated                                                                                                                                                     
    DEBUG  gateway/rep-0/GatewayRuntime@42053 Received signal SIGTERM                                                                                                                                                  
    DEBUG  gateway/rep-0@42053 process terminated                                                                                                                                                   [01/06/23 00:13:03]
    DEBUG  gateway/rep-0@42018 terminated                                                                                                                                                           [01/06/23 00:13:03]
    DEBUG  gateway/rep-0@42018 joining the process                                                                                                                                                                     
    DEBUG  gateway/rep-0@42018 successfully joined the process                                                                                                                                                         
    DEBUG  segmenter/rep-0@42018 waiting for ready or shutdown signal from runtime                                                                                                                  [01/06/23 00:13:03]
    DEBUG  segmenter/rep-0@42018 terminate                                                                                                                                                                             
    DEBUG  segmenter/rep-0@42018 terminating the runtime process                                                                                                                                                       
    DEBUG  segmenter/rep-0@42018 runtime process properly terminated                                                                                                                                                   
    
    
    

    I also note that this does not occur with every PDF. The PDF which is causing problems is attached. rg.25si055505.pdf

    Describe how you solve it


    Environment

    - jina 3.13.0
    - docarray 0.20.1
    - jcloud 0.1.6
    - jina-hubble-sdk 0.28.0
    - jina-proto 0.1.13
    - protobuf 4.21.12
    - proto-backend upb
    - grpcio 1.47.2
    - pyyaml 6.0
    - python 3.8.15
    - platform Linux
    - platform-release 4.15.0-200-generic
    - platform-version #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
    - architecture x86_64
    - processor x86_64
    - uid 185410937616950
    - session-id 60a8000a-8d89-11ed-af64-a8a15912fa36
    - uptime 2023-01-06T00:14:28.766910
    - ci-vendor (unset)
    - internal False
    * JINA_DEFAULT_HOST (unset)
    * JINA_DEFAULT_TIMEOUT_CTRL (unset)
    * JINA_DEPLOYMENT_NAME (unset)
    * JINA_DISABLE_UVLOOP (unset)
    * JINA_EARLY_STOP (unset)
    * JINA_FULL_CLI (unset)
    * JINA_GATEWAY_IMAGE (unset)
    * JINA_GRPC_RECV_BYTES (unset)
    * JINA_GRPC_SEND_BYTES (unset)
    * JINA_HUB_NO_IMAGE_REBUILD (unset)
    * JINA_LOG_CONFIG (unset)
    * JINA_LOG_LEVEL (unset)
    * JINA_LOG_NO_COLOR (unset)
    * JINA_MP_START_METHOD (unset)
    * JINA_OPTOUT_TELEMETRY (unset)
    * JINA_RANDOM_PORT_MAX (unset)
    * JINA_RANDOM_PORT_MIN (unset)
    * JINA_LOCKS_ROOT (unset)
    * JINA_K8S_ACCESS_MODES (unset)
    * JINA_K8S_STORAGE_CLASS_NAME (unset)
    * JINA_K8S_STORAGE_CAPACITY (unset)
    * JINA_STREAMER_ARGS (unset)
    
    opened by themantalope 3
Releases(v3.13.2)
  • v3.13.2(Dec 30, 2022)

    Release Note (3.13.2)

    This release contains 1 bug fix.

    ๐Ÿž Bug Fixes

    Respect timeout_ready when generating startup probe (#5560)

    As Kubernetes Startup Probes were added to all deployments in release v3.13.0, we added default values for all probe configurations. However, if those default configurations were not enough to wait for an Executor that takes time to load and become ready, the Executor deployment would become subject to the configured restart policy. Therefore, Executors that are slow to load would keep restarting forever.

    In this patch, this behavior is fixed by making sure that Startup Probe configurations respect the timeout_ready argument of Executors. Startup Probes are configured like so:

    • periodSeconds always set to 5 seconds
    • timeoutSeconds always set to 10 seconds
    • failureThreshold is the number of attempts maybe by kubernetes to check if the pod is ready. It varies according to timeout_ready. The formula used is failureThreshold = timeout_ready / 5000 (as timeout_ready is in microseconds and periodSeconds is 5 seconds) and in all cases, it will be at least 3. If timeout_ready is -1 (in Jina it means waiting forever for the Executor to become ready), since waiting forever is not supported in Kubernetes, the value for failureThreshold will be 120 attempts.

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.1(Dec 21, 2022)

    Release Note (3.13.1)

    Release time: 2022-12-21 12:58:15

    This release contains 3 bug fixes and 1 documentation improvement.

    ๐Ÿž Bug Fixes

    Support Gateway with multiple protocols for Kubernetes export (#5532)

    You can now export Flows with multiple protocols to Kubernetes. Previously this would cause an error.

    flow = Flow().config_gateway(protocol=['http', 'grpc'])
    flow.to_kubernetes_yaml('k8s_flow_folder')
    

    Fix Python 3.11 support (#5529)

    It was previously impossible to install Jina with Python 3.11 due to a grpcio dependency problem. grpcio added support for Python 3.11 only with version 1.49.0, causing potential problems when used by Jina and other projects.

    In this release grpcio>=1.49.0 is installed alongside Jina when using Python 3.11. However, be aware of potential problems related to grpc hanging.

    Unary RPC from Client respects results_in_order (#5513)

    In prior releases, calling the post method of a client with grpc and using stream=False did not respect the results_in_order parameter and results were always returned in order:

    # this wrongly returns results in order
    c = Client(protocol='grpc')
    c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)  
    

    Also this implied that using the Client with asyncio=True and stream=False in the post call would return results in the order that they were returned by the Flow, rather than respecting the input order:

    # this wrongly returns results in order
    c = Client(protocol='grpc', asyncio=True)
    
    async for resp in c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)
        print(resp)
    

    This release fixes the ordering bug.

    ๐Ÿ“— Documentation Improvements

    • Document inheritance of arguments from Flow API to Executors and Gateway (#5535)

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.0(Dec 15, 2022)

    Release Note (3.13.0)

    Release time: 2022-12-15 15:33:43

    This release contains 14 new features, 9 bug fixes and 7 documentation improvements.

    This release introduces major features like Custom Gateways, Dynamic Batching for Executors, development support with auto-reloading, support for the new namespaced Executor scheme jinaai, improvements for our gRPC transport layer, and more.

    ๐Ÿ†• Features

    Custom Gateways (#5153, #5189, #5342, #5457, #5465, #5472 and #5477)

    Jina Gateways are now customizable in the sense that you can implement them in much the same way as an Executor. With this feature, Jina gives power to the user to implement any server, protocol or interface at the Gateway level. There's no more need to build an extra service that uses the Flow.

    For instance, you can define a Jina Gateway that communicates with the rest of Flow Executors like so:

    from docarray import Document, DocumentArray
    from jina.serve.runtimes.gateway.http.fastapi import FastAPIBaseGateway
    
    
    class MyGateway(FastAPIBaseGateway):
        @property
        def app(self):
            from fastapi import FastAPI
    
            app = FastAPI(title='Custom FastAPI Gateway')
    
            @app.get(path='/service')
            async def my_service(input: str):
                # convert input request to Documents
                docs = DocumentArray([Document(text=input)])
    
                # send Documents to Executors using GatewayStreamer
                result = None
                async for response_docs in self.streamer.stream_docs(
                    docs=docs,
                    exec_endpoint='/',
                ):
                    # convert response docs to server response and return it
                    result = response_docs[0].text
    
                return {'result': result}
    
            return app
    

    Then you can use it in your Flow in the following way:

    flow = Flow().config_gateway(uses=MyGateway, port=12345, protocol='http')
    

    A Custom Gateway can be used as a Python class, YAML configuration or Docker image.

    Adding support for Custom Gateways required exposing the Gateway API and supporting multiple ports and protocols (mentioned in a prior release). You can customize it by subclassing the FastAPIBaseGateway class (for simple implementation) or base Gateway for more complex use cases.

    Working on this feature also involved exposing and improving the GatewayStreamer API as a way to communicate with Executors within the Gateway.

    Find more information in the Custom Gateway page.

    Dynamic batching (#5410)

    This release adds Dynamic batching capabilities to Executors.

    Dynamic batching allows requests to be accumulated and batched together before being sent to an Executor. The batch is created dynamically depending on the configuration for each endpoint.

    This feature is especially relevant for inference tasks where model inference is more optimized when batched to efficiently use GPU resources.

    You can configure Dynamic batching using either a decorator or the uses_dynamic_batching parameter. The following example shows how to enable Dynamic batching on an Executor that performs model inference:

    from jina import Executor, requests, dynamic_batching, Flow, DocumentArray, Document
    import numpy as np
    import torch
    
    
    class MyExecutor(Executor):
        def __init__(self, **kwargs):
            super().__init__(**kwargs)
    
            # initialize model
            self.model = torch.nn.Linear(in_features=128, out_features=128)
    
        @requests(on='/bar')
        @dynamic_batching(preferred_batch_size=10, timeout=200)
        def embed(self, docs: DocumentArray, **kwargs):
            docs.embeddings = self.model(torch.Tensor(docs.tensors))
    
    
    flow = Flow().add(uses=MyExecutor)
    

    With Dynamic Batching enabled, the Executor above will efficiently use GPU resources to perform inference by batching Documents together.

    Read more about the feature in the Dynamic Batching documentation page.

    Install requirements of local Executors (#5508)

    Prior to this release, the install_requirements parameter of Executors only installed Executor requirements for Hub Executors. Now, local Executors with a requirements.txt file will also have their requirements installed before starting Flows.

    Support jinaai Executor scheme to enable namespaced Hub Executors (#5462, #5468 and #5515)

    As Jina AI Cloud introduced namespaces to Executor resources, we made changes to support the new jinaai Executor scheme. This PR adds support for the new scheme.

    This means that namespaced Executors can now be used with the jinaai scheme in the following way:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai://jina-ai/DummyHubExecutor')
    

    This scheme is also supported in Kubernetes and other APIs:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai+docker://jina-ai/DummyHubExecutor')
    flow.to_kubernetes_yaml('output_path', k8s_namespace='my-namespace')
    

    The support of the new scheme means the minimum supported version of jina-hubble-sdk has been increased to 0.26.10.

    Add auto-reloading to Flow and Executor on file changes (#5461, #5488 and #5514)

    A new argument reload has been added to the Flow and Executor APIs, which automatically reloads running Flows and Executors when changes are made to Executor source code or YAML configurations of Flows and Executors.

    Although this feature is only meant for development, it aims to help developers iterate fast and automatically update Flows with changes they make live during development.

    Find out more about this feature in these two sections:

    Expand Executor serve parameters (#5494)

    The method Executor.serve can receive more parameters, similar to what the Flow API expects. With new parameters to control serving and deployment configurations of the Executor, this method empowers the Executor to be convenient for single service tasks.

    This means you can not only build advanced microservices-based pipelines and applications, but also build individual services with all Jina features: shards/replicas, dynamic batching, auto-reload, etc.

    Read more about the method in the Python API documentation.

    Add gRPC trailing metadata when logging gRPC error (#5512)

    When logging gRPC errors, context trailing metadata is now shown. This helps identify underlying network issues rather than the error codes that mask multiple network errors into a single gRPC status code.

    For instance, the new log message looks like the following:

    DEBUG  gateway@ 1 GRPC call to deployment executor0 failed                      
           with error <AioRpcError of RPC that terminated with:                     
                   status = StatusCode.UNAVAILABLE                                  
                   ...                                                                   
           trailing_metadata=Metadata((('content-length', '0'),                     
           ('l5d-proxy-error', 'HTTP Balancer service in                            
           fail-fast'), ('l5d-proxy-connection', 'close'),                          
           ('date', 'Tue, 13 Dec 2022 10:20:15 GMT'))), for                         
           retry attempt 2/3. Trying next replica, if available. 
    

    The trailing_metadata returned by load balancers will help to identify the root cause more accurately.

    Implement unary_unary stub for Gateway Runtime (#5507)

    This release adds the gRPC unary_unary stub for Gateway Runtime as a new communication stub with Executors. Since the gRPC performance best practices for Python page suggests that unary stream implementation might be faster for Python, we added this communication method.

    However, this is not enabled by default. The streaming RPC method will still be used unless you set the stream option to False in the Client.post() method. The feature is only effective when the gRPC protocol is used.

    Read more about the feature in the documentation: https://docs.jina.ai/concepts/client/send-receive-data/#use-unary-or-streaming-grpc

    Add Startup Probe and replace Readiness Probe with Liveness Probe (#5407)

    Before this release, when exporting Jina Flows to Kubernetes YAML configurations, Kubernetes Readiness Probes used to be added for the Gateway pod and each Executor pod. In this release we have added a Startup Probe and replaced Readiness Probe with Liveness Probe.

    Both probes use the jina ping command to check that pods are healthy.

    New Jina perf Docker images (#5498)

    We added a slightly larger Docker image with suffix perf which includes a set of tools useful for performance tuning and debugging.

    The new image is available in Jina AI's Docker hub.

    New Jina Docker image for Python 3.10, and use Python 3.8 for default Jina image (#5490)

    Besides adding Docker images aimed for performance optimization, we added an image with a newer Python version: 3.10. This is available in Jina AI's Docker hub, for example jinaai/jina:master-py310-standard.

    We also made Python 3.8 our minimum supported Python version by default, and it will be used for default Docker images.

    Minimize output of jina ping command (#5476)

    jina ping commands are now less verbose and will print less irrelevant output. However, important information like latency for each round, average latency, number of successful requests and ping result will still show up.

    Add Kubernetes preStop hook to the containers (#5445)

    A preStop hook has been added to Executors and the Gateway to allow a grace period. This allows more time to complete in-flight requests and finish the server's graceful shutdown.

    Generate random ports for multiple protocols (#5455)

    If you use multiple protocols for a Gateway, you no longer need to specify a port for each one. Whether it's Python or YAML, you just need to specify the protocols you want to support and Jina will generate random ports for you.

    Python API:

    from jina import Flow
    
    flow = Flow().config_gateway(protocol=['grpc', 'http', 'websocket'])
    with flow:
        flow.block()
    

    YAML:

    jtype: Flow
    gateway:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
    

    Result: flow

    ๐Ÿž Bug Fixes

    List-like args passed as string (#5464)

    We fixed the format expected for port, host and port_monitoring to feel more Pythonic. Basically, if you use replicas, you no longer have to provide comma-separated ports as a string value. Instead, you can simply pass a list of values, no need to put all in a string anymore!

    For instance, suppose we have two external replicas of an Executor that we want to join in our Flow (the first is hosted on localhost:12345 and the second on 91.198.174.192:12346). We can add them like this:

    from jina import Flow
    
    replica_hosts, replica_ports = ['localhost', '91.198.174.192'], [
        '12345',
        '12346',
    ]  # instead of 'localhost,91.198.174.192', '12345,12346'
    Flow().add(host=replica_hosts, port=replica_ports, external=True)
    

    Or:

    Flow().add(host=['localhost:12345', '91.198.174.192:12346'], external=True)
    

    Note that this is not a breaking change, and the old syntax (comma-separated values: Flow().add(host='localhost:12345,91.198.174.192:12346', external=True)) is still supported for backwards compatibility.

    Restore port to overload type hint and JSON schema (#5501)

    When we made port and protocol arguments of the Gateway support multiple values, a bug was introduced where port did not appear in Jina's JSON schema as well as the Flow API overload for method signatures.

    Although the arguments are functional in both the Python API and YAML, this suppressed auto-completion and developer support for these parameters. This release restores the port parameter in both the Flow method overloads and JSON schema.

    Do not force insecure to True in open telemetry integration (#5483)

    In Jina's instrumentation, communication to open telemetry exporters used to be forced to insecure mode. Luckily, our community member @big-thousand picked this up and submitted a fix. The communication is no longer forced to the insecure mode.

    Kudos to @big-thousand for his contribution!

    Fix problem when using floating Executor in HTTP (#5493)

    We found a bug when using Floating Executors in HTTP, where the floating Executor is connected to the Gateway (in the Flow topology). In this case, the Executor would not receive input Documents properly. This release fixes the mentioned bug.

    Add egg info post install command for egg info setup mode (#5491)

    This release adds support for the egg info setup mode in Python. This means post-installation commands are now properly executed in environments that rely on Python's new setup mode.

    This bug resulted in several issues especially for environments that depend on these post-installation commands. For instance, some Environment Variables that are needed for Jina to work on macOS and for CLI auto-complete.

    Do not apply limits when gpus='all' in Kubernetes (#5485)

    If Executor parameter gpus is set to "all", no limits will be applied on the pod in Kubernetes.

    Fix Windows signal handling (#5484)

    This release improves signal handling on Windows, specifically when cancelling a Flow with an OS signal.

    Cap opentelemetry-instrumentation-aiohttp-client (#5452)

    This release caps the version for opentelemetry-instrumentation-aiohttp-client which is incompatible with opentelemetry-semantic-conventions.

    Raise exceptions from path importer (#5447)

    Previously, errors were hidden when they came from a Python module imported to load an Executor. Actually the module was not considered to be a Python module, which produced other misleading errors. In this release, actual errors during imports will be raised and no longer hidden.

    ๐Ÿ“— Documentation Improvements

    • Add gRPC requirements for Apple Silicon (M1 Chip) to fix failing installation of Jina (#5511)
    • Add redirects from '/fundamentals' to '/concepts' (#5504)
    • Update JCloud documentation to the jcloud v0.1.0 (#5385)
    • Restructure documentation under /concepts
    • Change Executor URI scheme to namespaced scheme jinaai (#5450)
    • Custom Gateway documentation (#5465)
    • Provide more accurate description for port and protocol parameters of the Gateway (#5456)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Delgermurun (@delgermurun)
    • Jie Fu (@jemmyshin)
    • Alex Cureton-Griffiths (@alexcg1)
    • big-thousand (@big-thousand)
    • IyadhKhalfallah (@IyadhKhalfallah)
    • Deepankar Mahapatro (@deepankarm)
    • samsja (@samsja)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Joan Fontanals (@JoanFM)
    • Anne Yang (@AnneYang720)
    • Han Xiao (@hanxiao)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    Source code(tar.gz)
    Source code(zip)
  • v3.12.0(Nov 25, 2022)

    Release Note (3.12.0)

    Release time: 2022-11-25 12:28:29

    This release contains 8 new features, 16 bug fixes and 15 documentation improvements.

    ๐Ÿ†• Features

    Support multiple protocols at the same time in Flow Gateways (#5435 and #5378)

    Prior to this release, a Flow only exposed one server in its Gateway with one of the following protocols: HTTP, gRPC or WebSockets.

    Now, you can specify multiple protocols and for each one, a separate server is started. Each server is bound to its own port.

    For instance, you can do:

    from jina import Flow
    flow = Flow(port=[12345, 12346, 12347], protocol=['http', 'grpc', 'websocket'])
    with flow:
        flow.block()
    

    or: jina flow --uses flow.yml where flow.yml is:

    jtype: Flow
    with:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
      port:
        - 12345
        - 12344
        - 12343
    

    multi-protocol-flow

    The protocol and port parameters can still accept single values rather than a list. Therefore, there is no breaking change.

    Alias parameters protocols and ports are also defined:

    flow = Flow(ports=[12345, 12346, 12347], protocols=['http', 'grpc', 'websocket'])
    

    In Kubernetes, this exposes separate services for each protocol.

    Read the docs for more information.

    Add Kubernetes information to resource attributes in instrumentation (#5372)

    When deploying to Kubernetes, the Gateway and Executors expose the following Kubernetes information as resource attributes in instrumentation:

    • k8s.namespace.name
    • k8s.pod.name
    • k8s.deployment.name / k8s.statefulset.name

    Besides that, the following resource attributes are set if they are present in the environment variables of the container:

    • k8s.cluster.name (set K8S_CLUSTER_NAME environment variable)
    • k8s.node.name (set K8S_NODE_NAME environment variable)

    Add option to return requests in order using the Client (#5404)

    If you use replicated Executors, those which finish processing first return their results to the Gateway which then returns them to the client. This is useful if you want results as soon as each replicated Executor finishes processing your Documents.

    However, this may be inconvenient if you want the Documents you send to the Flow to return in order. In this release, you can retain the order of sent Documents (when using replicated Executors) by passing the results_in_order parameter in the Client.

    For instance, if your Flow looks like this:

    from jina import Flow, DocumentArray, Document
    f = Flow().add(replicas=2)
    

    You can do the following to keep results in order:

    input_da = DocumentArray([Document(text=f'{i}') for i in range(100)])
    
    with f:
        result_da = f.post('/', inputs=input_da, request_size=10,  results_in_order=True)
        assert result_da[:, 'text'] == input_da[:, 'text']
    

    Add docs_map parameter to Executor endpoints (#5366)

    Executor endpoint signatures are extended to the following:

    class MyExecutor(Executor):
        @requests
        async def foo(
            self, docs: DocumentArray, parameters: Dict, docs_matrix: Optional[List[DocumentArray]], docs_map: Optional[Dict[str, DocumentArray]]
        ) -> Union[DocumentArray, Dict, None]:
            pass
    

    Basically, the parameter docs_map has been added. It's a dictionary that maps previous Executor names to DocumentArrays. This is useful when you have an Executor that combines results from many previous Executors, and you need information about where each resulting DocumentArray comes from.

    Add Gateway API (#5342)

    Prior to this release, all Gateway configurations were specified in the Flow API. However, by principle, Flow parameters are commonly inherited by Executors and the Gateway. We already gave the Executor its own API to be customized (either using the method add() or the executors YAML section in Flow YAML).

    In this release, we have done the same for Gateway. It defines its own API in both the Python API and YAML interface. In the Python API, you can configure the Gateway using the config_gateway() method:

    flow = Flow().config_gateway(port=12345, protocol='http')
    

    And in the YAML interface, you can configure the Gateway using the gateway section:

    !Flow
    gateway:
      protocol: http
      port: 12344
    executors:
      - name: exec
    

    This is useful when you want to apply parameters just for the Gateway. If you want a parameter to be applied to all Executors, then continue to use the Flow API.

    Keep in mind that you can still provide Gateway parameters using the Flow API. This means there are no breaking changes introduced.

    Support UUID in CUDA_VISIBLE_DEVICES round-robin assignment (#5360)

    You can specify a comma-separated list of GPU UUIDs in the CUDA_VISIBLE_DEVICES to assign devices to Executor replicas in a round-robin fashion. For instance:

    CUDA_VISIBLE_DEVICES=RRGPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5,GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5,GPU-0ccccccc-74d2-7297-d557-12771b6a79d5,GPU-0ddddddd-74d2-7297-d557-12771b6a79d5
    

    Check CUDA's documentation to see the accepted formats to assign CUDA devices by UUID.

    | GPU device | Replica ID | |------------------------------------------|------------| | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 0 | | GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5 | 1 | | GPU-0ccccccc-74d2-7297-d557-12771b6a79d5 | 2 | | GPU-0ddddddd-74d2-7297-d557-12771b6a79d5 | 3 | | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 4 |

    Thanks to our community member @mchaker for submitting this feature request!

    Capture shard failures in the head runtime (#5338)

    In case you use Executor shards, partially failed requests (those that fail on a subset of the shards) no longer raise an error.

    Instead, successful results are returned. An error is raised only when all shards fail to process Documents. Basically, the HeadRuntime's behavior is updated to fail only when all shards fail.

    Thanks to our community user @soumil1 for submitting this feature request.

    Add successful, pending and failed metrics to HeadRuntime (#5374)

    More metrics have been added to the Head Pods:

    • jina_number_of_pending_requests: number of pending requests
    • jina_successful_requests: number of successful requests
    • jina_failed_requests: number of failed requests
    • jina_received_request_bytes: the size of received requests in bytes
    • jina_sent_response_bytes: the size of sent responses in bytes

    See more in the instrumentation docs.

    Add deployment label in grpc stub metrics (#5344)

    Executor metrics used to show up aggregated at the Gateway level and users couldn't see separate metrics per Executor. With this release, we have added labels for Executors so that metrics in the Gateway can be generated per Executor or aggregated over all Executors.

    ๐Ÿž Bug Fixes

    Check whether the deployment is in Executor endpoints mapping (#5440)

    This release adds an extra check in the Gateway when sending requests to deployments: The Gateway sends requests to the deployment only if it is in the Executor endpoint mapping.

    Unblock event loop to allow health service (#5433)

    Prior to this release, sync function calls inside Executor endpoints blocked the event loop. This meant that health-checks submitted to Executors failed for long tasks (for instance, inference using a large model).

    In this release, such tasks no longer block the event loop. While concurrent requests to the same Executor wait until the sync task finishes, other runtime tasks remain functional, mainly health-checks.

    Dump environment variables to string for Kubernetes (#5430)

    Environment variables are now cast to strings before dumping them to Kubernetes YAML.

    Unpin jina-hubble-sdk version (#5412)

    This release frees (unpins) jina-hubble-sdk version. The latest jina-hubble-sdk is installed with the latest Jina.

    Bind servers to host argument instead of __default_host__ (#5405)

    This release makes servers at each Jina pod (head, Gateway, worker) bind to the host address specified by the user, instead of always binding to the __default_host__ corresponding to the OS. This lets you, depending on your network interface, restrict or expose your Flow services in your network.

    For instance, if you wish to expose all pods to the internet, except for the last Executor, you can do:

    flow = Flow(host='0.0.0.0').add().add(host='127.0.0.1')
    

    After this fix, Jina respects this syntax and binds the last Executor only to 127.0.0.1 (accessible only inside the host machine).

    Thanks to @wqh17101 for reporting this issue!

    Fix backoff_multiplier format when using max_attempts in the Client (#5403)

    This release fixes the format of backoff_multiplier parameter when injected into the gRPC request. The issue appeared when you use the max_attempts parameter in Client.

    Maintain the correct tracing operations chain (#5391)

    Tracing spans for Executors used to show up out of order. This behavior has been fixed by using the method start_as_current_span instead of start_span to maintain the tracing chain in the correct order.

    Use Async health servicer for tracing interceptors when tracing is enabled (#5392)

    When tracing is enabled, health checks in Docker and Kubernetes deployments used to fail silently until the Flow timed out. This happened because tracing interceptors expected RPC stubs to be coroutines.

    This release fixes this issue by using the async aio.HealthServicer instead of grpc_health.HealthServicer. Health checks submitted to runtimes (Gateway, head, worker) no longer fail when tracing is enabled.

    Properly update requests count in case of Exception inside the HeadRuntime (#5383)

    In case of an Exception being raised in the HeadRuntime, request counts were not updated properly (pending requests should have been decremented and failed requests should have been incremented). This is fixed in this release and the Exception is caught to update request counts.

    Fix endpoint binding when inheriting Executors (#5380)

    When an Executor is inherited, the bound endpoints of the parent Executor used to be overridden by those of the child Executor. This meant, if you inherited from Executors but still chose to use the parent Executor in your Flow, a wrong endpoint could have been called. This behavior is fixed by making Executor.requests a nested dict that also includes information about the class name. This helps to properly support Executor inheritance.

    Missing recording logic in connection stub metrics (#5363)

    Recording of request and response size in bytes is fixed to track all cases. This makes these metrics more accurate for the Gateway.

    Move build configs to pyproject (#5351)

    Build requirements have been moved from setup.py to pyproject.toml. This suppresses deprecation warnings that show up when installing Jina.

    New timer should keep labels (#5341)

    The MetricsTimer in instrumentation previously created new timers without keeping the histogram metric labels. This behavior is fixed and new timers retain the same labels.

    Use non-mutable default for MetricsTimer constructor (#5339)

    Use None instead of empty dict as a default value for histogram_metric_labels in MetricsTimer constructor.

    Catch RpcErrors and show better error messages in Client (#5325)

    In the Client, we catch RpcError and show its details instead of showing a standard error message.

    Import OpenTelemetry functions only when tracing is enabled in WorkerRuntime (#5321)

    This release ensures OpenTelemetry functions are only imported when tracing is enabled in the worker.

    ๐Ÿ“— Documentation Improvements

    • Remove 3 off-topic articles
    • Enable flag to convert resource labels to metric labels (#5409)
    • Add reference to Jina Kotlin client from community (#5390)
    • Add section about the transition from DocArray to Jina (#5382)
    • Add tips about supporting multiprocessing with fork in Jina when using macOS (#5379)
    • Fix reference to multiprocessing with spawn section and emphasize the need for entrypoint protection (#5370)
    • Update instructions to build protos locally using protogen Docker image (#5335)
    • Change mentions of JCloud to Jina AI Cloud (#5329)
    • Restructure docs into cloud-native section (#5332)
    • Add contributor acknowledgement and spell checking (#5324)
    • Create a Kubernetes section (#5315)
    • Add reference to Go and PHP clients (#5253)
    • Introduce versioning to the documentation (#5310)
    • Support redirects for removed documentation pages (#5301)
    • Use better Grafana screenshot without random text block (#5306)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Ziniu Yu (@ZiniuYu)
    • Andrei Ungureanu (@Andrei997)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Joan Fontanals (@JoanFM)
    • Zhaofeng Miao (@mapleeit)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    • Han Xiao (@hanxiao)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Nan Wang (@nan-wang)
    Source code(tar.gz)
    Source code(zip)
  • v3.11.2(Nov 22, 2022)

    Release note

    This release contains 1 hot fix.

    ๐Ÿž Bug Fix

    Unpin jina-hubble-sdk

    To avoid conflicting dependency versions with docarray, unpin the jina-hubble-sdk version dependency

    ๐ŸคŸ Contributors

    Joan Fontanals (@JoanFM )

    Source code(tar.gz)
    Source code(zip)
  • v3.11.0(Oct 24, 2022)

    Release Note (3.11.0)

    Release time: 2022-10-24 14:42:35

    This release contains 6 new features, 1 bug fix and 10 documentation improvements.

    ๐Ÿ†• Features

    Add OpenTelemetry tracing and metrics with basic configuration (#5175)

    Jina now supports OpenTelemetry Tracing and Metrics libraries for increased observability and instrumentation of Jina Runtimes. Read the docs (#5291) for details. We also provide a migration guide for a smooth transition from the previously-supported Prometheus-only based metrics instrumentation.

    You can integrate Jaeger or other distributed tracing tools to collect and visualize request-level and application level service operation attributes. This helps you analyze request-response lifecycle, application behavior and performance.

    from jina import Executor, requests, DocumentArray
    
    class MyExec(Executor):
        @requests
        def encode(self, docs: DocumentArray, **kwargs):
            with self.tracer.start_as_current_span(
                'encode', context=tracing_context
            ) as span:
                with self.monitor(
                    'preprocessing_seconds', 'Time preprocessing the requests'
                ):
                    docs.tensors = preprocessing(docs)
                with self.monitor(
                    'model_inference_seconds', 'Time doing inference the requests'
                ):
                    docs.embedding = model_inference(docs.tensors)
    

    Record existing Prometheus metrics into OpenTelemetry histograms (#5275)

    Prometheus-only based metrics are also available as OpenTelemetry supported metrics. You can switch to OpenTelemetry metrics with little effort.

    Add default tracing interceptors to head gRPC connection pool (#5271)

    Head Runtime supports default traces when forwarding requests to the shard replica.

    Add gRPC metadata to Executors and Gateway deployments (#5221)

    You can now provide HTTP header access tokens as client metadata for accessing external and authentication protected services.

    from jina import Client
    
    client = Client(host='api.clip.jina.ai', port=2096, tls=True, protocol='grpc')
    client.post(on='/encode', metadata=(('authorization', '<your access token>'),))
    

    Show local location of Executors in Hub (#5282)

    Use the list sub command to list the locations of local Hub Executors:

    jina hub list
    

    asciicast

    Dump to Statefulset in K8s when volumes are passed to Executor (#5265)

    Jina will generate a Kubernetes Statefulset manifest for Executors if you provide volumes to the Flow.to_kubernets_yaml() method. Executors can persist data on mounted volumes and correctly scale horizontally if required.

    ๐Ÿž Bug Fixes

    Invalid input raises exception (#5141)

    • Theclient.post() method was not raising an exception if any of the underlying async requests raised an exception.
    • The new version correctly raises the exception raised by any async request. The user can now correctly catch any exceptions and act upon the exception.

    ๐Ÿ“— Documentation Improvements

    • Docs for OpenTelemetry instrumentation (#5291)
    • Add Jina AI Cloud restructure Jcloud and Hub (#5298)
    • Standardize naming conventions (#5285)
    • Fix metrics naming (#5277)
    • Misc clean up (#5255, #5287, #5284, #5290, #5289, #5288)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Anthony Le (@AnthonyLe93)
    • felix-wang (@numb3r3)
    • Girish Chandrashekar (@girishc13)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Han Xiao (@hanxiao)
    • Alex Cureton-Griffiths (@alexcg1)
    • Andrei Ungureanu (@Andrei997)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.1(Oct 6, 2022)

    Release Note (3.10.1)

    Release time: 2022-10-06 15:05:48

    This release contains 1 new feature, 1 refactor, 3 bug fixes and 7 documentation improvements.

    ๐Ÿ†• Features

    Distributed replicas across different hosts (#5217)

    You can now start different Executors on different machines and use them as replicas by passing their addresses in the host parameter as a list, together with the external flag set to True:

    f.add(host='localhost:12345,91.198.174.192:12346', external=True)
    

    The Flow ensures the requests are load balanced between Executor instances.

    โš™ Refactoring

    Move Hubble related code to hubble-sdk (#5227)

    All code related to hubble and communication with Jina Hub has been moved to the jina-hubble-sdk repo and has been added as an external dependency.

    ๐Ÿž Bug Fixes

    Disable rich traceback in logging (#5242)

    We disabled the use of rich logging when printing tracebacks for Exceptions. This makes it easier to debug issues as the lines are not broken.

    Previously, an Exception would be reported to the console as:

      ERROR  executor0/rep-0@183551 ZeroDivisionError('division by [10/05/22 11:26:23]
           zero')                                                                   
            add "--quiet-error" to suppress the exception                           
           details                                                                  
           โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Traceback (most recent call last) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in process_data                                   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   179 โ”‚   โ”‚   โ”‚   โ”‚   if self.logger.debug_enable โ”‚                    
           โ”‚   180 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._log_data_request( โ”‚                    
           โ”‚   181 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚ โฑ 182 โ”‚   โ”‚   โ”‚   โ”‚   result = await self._data_r โ”‚                    
           โ”‚   183 โ”‚   โ”‚   โ”‚   โ”‚   if self._successful_request โ”‚                    
           โ”‚   184 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._successful_reques โ”‚                    
           โ”‚   185 โ”‚   โ”‚   โ”‚   โ”‚   return result               โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in handle                                         โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   161 โ”‚   โ”‚   )                                   โ”‚                    
           โ”‚   162 โ”‚   โ”‚                                       โ”‚                    
           โ”‚   163 โ”‚   โ”‚   # executor logic                    โ”‚                    
           โ”‚ โฑ 164 โ”‚   โ”‚   return_data = await self._executor. โ”‚                    
           โ”‚   165 โ”‚   โ”‚   โ”‚   req_endpoint=requests[0].header โ”‚                    
           โ”‚   166 โ”‚   โ”‚   โ”‚   docs=docs,                      โ”‚                    
           โ”‚   167 โ”‚   โ”‚   โ”‚   parameters=params,              โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall__                                      โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   290 โ”‚   โ”‚   if req_endpoint in self.requests:   โ”‚                    
           โ”‚   291 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   292 โ”‚   โ”‚   elif __default_endpoint__ in self.r โ”‚                    
           โ”‚ โฑ 293 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   294 โ”‚                                           โ”‚                    
           โ”‚   295 โ”‚   async def __acall_endpoint__(self, req_ โ”‚                    
           โ”‚   296 โ”‚   โ”‚   func = self.requests[req_endpoint]  โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall_endpoint__                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   311 โ”‚   โ”‚   โ”‚   if iscoroutinefunction(func):   โ”‚                    
           โ”‚   312 โ”‚   โ”‚   โ”‚   โ”‚   return await func(self, **k โ”‚                    
           โ”‚   313 โ”‚   โ”‚   โ”‚   else:                           โ”‚                    
           โ”‚ โฑ 314 โ”‚   โ”‚   โ”‚   โ”‚   return func(self, **kwargs) โ”‚                    
           โ”‚   315 โ”‚                                           โ”‚                    
           โ”‚   316 โ”‚   @property                               โ”‚                    
           โ”‚   317 โ”‚   def workspace(self) -> Optional[str]:   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in arg_wrapper                                    โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   159 โ”‚   โ”‚   โ”‚   โ”‚   def arg_wrapper(            โ”‚                    
           โ”‚   160 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   executor_instance, *arg โ”‚                    
           โ”‚   161 โ”‚   โ”‚   โ”‚   โ”‚   ):  # we need to get the su โ”‚                    
           โ”‚       the self                                    โ”‚                    
           โ”‚ โฑ 162 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   return fn(executor_inst โ”‚                    
           โ”‚   163 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚   164 โ”‚   โ”‚   โ”‚   โ”‚   self.fn = arg_wrapper       โ”‚                    
           โ”‚   165                                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/USER/.config/JetBrains/PyCharmCE2022.2/scrโ€ฆ โ”‚                    
           โ”‚ in foo                                            โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   12 โ”‚                                            โ”‚                    
           โ”‚   13 โ”‚   @requests                                โ”‚                    
           โ”‚   14 โ”‚   def foo(self, docs, **kwargs):           โ”‚                    
           โ”‚ โฑ 15 โ”‚   โ”‚   1/0                                  โ”‚                    
           โ”‚   16                                              โ”‚                    
           โ”‚   17 with Flow().add(uses=MyExec) as f:           โ”‚                    
           โ”‚   18                                              โ”‚                    
           โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ                    
           ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
      File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
        f.search(inputs=[Document()])
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
        return run_async(
      File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
        return asyncio.run(func(*args, **kwargs))
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
        async for resp in c._get_results(*args, **kwargs):
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
        callback_exec(
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
        raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "c50a57014fb948ccbd8065ada487a136"
    status {
      code: ERROR
      description: "ZeroDivisionError(\'division by zero\')"
      exception {
        name: "ZeroDivisionError"
        args: "division by zero"
        stacks: "Traceback (most recent call last):\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
        stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
        stacks: "ZeroDivisionError: division by zero\n"
        executor: "MyExec"
      }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    After the fix, the error is much clearer:

    ERROR  executor0/rep-0@198650 ZeroDivisionError('division by [10/05/22 11:40:37]
         zero')                                                                   
          add "--quiet-error" to suppress the exception                           
         details                                                                  
         Traceback (most recent call last):                                       
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 182, in process_data                                                
             result = await                                                       
         self._data_request_handler.handle(requests=requests)                     
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 164, in handle                                                      
             return_data = await self._executor.__acall__(                        
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 293, in __acall__                                                   
             return await                                                         
         self.__acall_endpoint__(__default_endpoint__,                            
         **kwargs)                                                                
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 314, in __acall_endpoint__                                          
             return func(self, **kwargs)                                          
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 162, in arg_wrapper                                                 
             return fn(executor_instance, *args, **kwargs)                        
           File                                                                   
         "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratcโ€ฆ                    
         line 15, in foo                                                          
             1/0                                                                  
         ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
    File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
      f.search(inputs=[Document()])
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
      return run_async(
    File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
      return asyncio.run(func(*args, **kwargs))
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
      return loop.run_until_complete(main)
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
      return future.result()
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
      async for resp in c._get_results(*args, **kwargs):
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
      callback_exec(
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
      raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "5d5e0338d0cd4bdd89efc430bbaaa74d"
    status {
    code: ERROR
    description: "ZeroDivisionError(\'division by zero\')"
    exception {
      name: "ZeroDivisionError"
      args: "division by zero"
      stacks: "Traceback (most recent call last):\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
      stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
      stacks: "ZeroDivisionError: division by zero\n"
      executor: "MyExec"
    }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    Remove lambda function in logger (#5249)

    We removed a reference to a lambda function in the JinaLogger so that there is no problem pickling it:

    import pickle
    
    from jina import Executor
    
    class MyExecutor(Executor):
    
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
    
    executor = MyExecutor()
    
    with open('executor.pickle', 'wb') as f:
        pickle.dump(executor, f)
    

    Previously, an Executor could not be pickled because it had a reference to a JinaLogger.

    This used to raise an AttributeError exception:

    Traceback (most recent call last):
      File "/home/USER/toy.py", line 14, in <module>
        pickle.dump(executor, f)
    AttributeError: Can't pickle local object 'JinaLogger.__init__.<locals>.<lambda>'
    

    Fix gRPC fork support (#5250)

    We enabled proper fork support for gRPC to correctly clean up gRPC usage in parent and child processes. This prevents potential issues when forking Executor processes.

    ๐Ÿ“— Documentation Improvements

    • Explain Executor patching in Kubernetes (#5235)
    • Add note about keeping same Jina versions across microservices (#5239)
    • Add missing request numbers in sample code (#5237)
    • JCloud YAML reorder (#5234)
    • Docs spelling/grammar/punctuation (#5244) (#5240)
    • Fix typos (#5236)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Sami Jaghouar (@samsja)
    • Alex Cureton-Griffiths (@alexcg1)
    • Johannes Messner (@JohannesMessner)
    • Delgermurun (@delgermurun)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Jackmin801 (@Jackmin801)
    • Joan Fontanals (@JoanFM)
    • Girish Chandrashekar (@girishc13)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.0(Sep 29, 2022)

    Release Note (v3.10.0)

    Release time: 2022-09-29 16:18:19

    This release contains 10 new features, 9 bug fixes and 12 documentation improvements.

    ๐Ÿ†• Features

    Enable jina ping as readiness probe on local and Kubernetes (#5200)

    This feature lets you check the readiness of an entire Flow or its individual components:

    jina ping flow grpc://localhost:12345
    
    jina ping executor localhost:12346
    
    jina ping gateway grpc://localhost:12345
    

    Successful output:

    INFO   JINA@28600 readiness check succeeded 1 times!!! 
    

    Unsuccessful output:

    This command exits with code 1 when the readiness check is not successful. Therefore, it is a good choice to use as a readiness probe for Executor and Gateway when deployed in Kubernetes.

    Add argument to specify exceptions that will exit the runtime (#5165)

    You can specify which Exceptions should trigger termination of an affected Executor:

    Flow().add(uses=MyExecutor, exit_on_exceptions: ['Exception',  'RuntimeException'])
    

    This ensures an Executor is not stuck in an unusable but technically alive state, and that the Executor can be restarted. This is especially useful when Executor restarts are automated through Kubernetes.

    Retry mechanism for client.post (#5176)

    client.post() accepts max_attempts, initial_backoff, max_backoff and backoff_multiplier parameters to control the capacity to retry requests, when a transient connectivity error occurs, using an exponential backoff strategy:

    client.post(
        on='/',
        inputs=docs,
        max_attempts=5,
        initial_backoff=0.1,
        max_backoff=0.5,
        backoff_multiplier=1.5
    )
    

    Add monitoring of requests size everywhere (#5111)

    The size of requests (jina_request_size_bytes) is now monitored everywhere: On every Executor, every Gateway, and every sharding Head.

    Consequently, this new metric can now be accessed via Prometheus and Grafana.

    JSON logging for JCloud (#5201)

    It is now possible to create logs in JSON format, instead of the standard line-by-line text output.

    Sample output:

    {"created": 1663930387.5850368, "filename": "data_request_handler.py", "funcName": "_load_executor", "levelname": "DEBUG", "lineno": 98, "module": "data_request_handler", "msg": "<jina.serve.executors.BaseExecutor object at 0x7fc1951f78d0> is successfully loaded!", "name": "WorkerRuntime", "pathname": "/home/foo/jina/serve/runtimes/request_handlers/data_request_handler.py", "process": 13386, "processName": "Pod", "thread": 140469860276032, "threadName": "MainThread"}
    

    To enable this, set the environment variable 'JINA_LOG_CONFIG to 'json'.

    Support list-like syntax to specify visible cuda devices (#5187)

    You can now use device id's to specify visible CUDA devices that should be used in the round-robin GPU assignment:

    `CUDA_VISIBLE_DEVICES=RR1,3`
    

    This will assign CUDA devices 1 and 3 to the different Executor replicas in a round-robin fashion.

    Make jina hub push non-blocking (#5129)

    jina hub push is now a non-blocking operation, meaning that the user does not have to wait for the entire push to be completed on the server side.

    While the push is being processed in the background, its status can be checked with the following command:

    jina hub status [<path_to_executor_folder>] [--id TASK_ID] [--verbose] [--replay]
    

    Warning messages in jina hub push (#5156)

    jina hub push can now display helpful warning messages, such as deprecation warnings: Screen Shot 2022-09-09 at 16 32 58

    Add duration info in events (#5157)

    Telemetry data now also includes the duration of every event that is reported.

    ๐Ÿž Bug Fixes

    Fix compatibility with Protobuf Python backend (#5222)

    This bug caused an exception when Python was used as a backend for Protobuf, rather than cpp or udp.

    With this fix, Python can also be used as a backend for Protobuf.

    Move significant params to sequential (#5205)

    This bug (occasionally) caused docker.errors.NullResource: Resource ID was not provided when pulling from Jina Hub. This was due to a faulty local caching implementation, which is now fixed.

    Fix missing secret when logged-in user with --force-update (#5180)

    This bug caused a failure for logged in users when force pushing to Jina Hub: Their secret would not be passed. This is now fixed and force push works as expected.

    Use default gRPC parameters for gRPC connection pool connection (#5211)

    We brought our gRPC client in line with the recommended configuration at https://github.com/grpc/grpc/blob/master/doc/keepalive.md

    Bump DocArray version for new column syntax (#5171)

    Some changes to the column syntax in DocArray required a version bump.

    Increase minimum Protobuf version (#5166)

    Some changes in the Protobuf library required a version bump.

    Remove leftover prints (#5199)

    Some unwanted print statements had sneaked their way into our codebase, and we kicked them right back out!

    ๐Ÿ“— Documentation Improvements

    • Change setenv into environ (#5223)
    • Mention prefetch (#5220)
    • Fix install instructions docker (#5213)
    • Fix grammar, wording, punctuation (#5209)
    • Fix grammar, punctuation (#5208)
    • Clarify exec endpoint usage in http (#5202)
    • Document grpc client limitation (#5193)
    • Update faq and lifetime (#5191)
    • Improve warning about config file in custom docker (#5190)
    • Clarify return behavior in parameters (#5179)
    • Add section for exit_on_exceptions argument (#5172)
    • Labels in flow yaml (#5164)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Girish Chandrashekar (@girishc13)
    • Andrei Ungureanu (@Andrei997)
    • Michael Gรผnther (@guenthermi)
    • Deepankar Mahapatro (@deepankarm)
    • Han Xiao (@hanxiao)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Nikolas Pitsillos (@npitsillos)
    • samsja (@samsja)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.3(Sep 29, 2022)

    Release Note (3.9.3)

    Release time: 2022-09-29 16:05:04

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, AlaeddineAbdessalem, Michael Gรผnther, Yanlong Wang, samsja, Alex Cureton-Griffiths, Deepankar Mahapatro, Nikolas Pitsillos, Han Xiao, zhangkai, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [11533962] - add gateway option to jina ping and use ping in readinessProbe (#5200) (Joan Fontanals)
    • [d0838d37] - add multiple attempts options to client.post API (#5176) (Joan Fontanals)
    • [13edf908] - use default grpc parameters for grpc connection pool connection (#5211) (samsja)
    • [216b4bf0] - monitoring: add monitoring of requests size in bytes at all level (#5111) (samsja)
    • [737875f5] - logs: json logging for jcloud (#5201) (Deepankar Mahapatro)
    • [fd4c0347] - support list-like syntax to round robin CUDA devices (#5187) (Joan Fontanals)
    • [9dff4c88] - add duration info in events (#5157) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [3683fccf] - fix compatibility with protobuf python backend (#5222) (AlaeddineAbdessalem)
    • [e932d6f9] - hubio.fetch_meta: move significant params to sequential (#5205) (Yanlong Wang)
    • [60e9b8de] - remove leftover prints (#5199) (Joan Fontanals)
    • [da186a23] - provide logger and streamer (#5188) (AlaeddineAbdessalem)
    • [7b5c0316] - fix get-openapi-schemas script (#5185) (AlaeddineAbdessalem)
    • [caf4a3d6] - fix missing secret when logged-in user with --force-update and โ€ฆ (#5180) (zhangkai)

    ๐Ÿงผ Code Refactoring

    • [9a6079ef] - separate gateway and asyncio runtime readiness checks (#5224) (AlaeddineAbdessalem)
    • [243639dd] - extract gateway app logic into custom gateway class (#5153) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [147317db] - change setenv into environ (#5223) (Michael Gรผnther)
    • [ad37bdd2] - mention prefetch (#5220) (Joan Fontanals)
    • [b9c8e44e] - fix install instructions docker (#5213) (Joan Fontanals)
    • [7cbf3471] - what-is-jina: fix grammar, wording, punctuation (#5209) (Alex Cureton-Griffiths)
    • [7c7d2cb0] - what-is-modality: fix grammar, punctuation (#5208) (Alex Cureton-Griffiths)
    • [1399e36c] - clarify exec endpoint usage in http (#5202) (Joan Fontanals)
    • [842a585b] - document grpc client limitation (#5193) (AlaeddineAbdessalem)
    • [96c5e7c1] - jcloud: update faq and lifetime (#5191) (Nikolas Pitsillos)
    • [c17e6416] - improve warning about config file in custom docker (#5190) (Joan Fontanals)
    • [22330fcb] - clarify return behavior in parameters (#5179) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [b5977f47] - update contributing to remove labeling (#5225) (Joan Fontanals)
    • [45c7c6e1] - update pull request template link (#5214) (Joan Fontanals)
    • [1eab9ce6] - fix doc template (Han Xiao)
    • [8c2d0758] - docs: update TOC (Jina Dev Bot)
    • [139b068e] - version: the next version will be 3.9.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.2(Sep 15, 2022)

    Release Note (3.9.2)

    Release time: 2022-09-15 14:00:36

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Girish Chandrashekar, zhangkai, AlaeddineAbdessalem, Joan Fontanals, Jina Dev Bot, Andrei Ungureanu, Deepankar Mahapatro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [c51f9014] - hubble async push (#5129) (zhangkai)
    • [3f39ed46] - runtime: add argument to specify exceptions that will exit the runtime (#5165) (Girish Chandrashekar)
    • [27e1f779] - hubio: display warning messages from hubble request (jina hub push) (#5156) (Andrei Ungureanu)

    ๐Ÿž Bug fixes

    • [29ad1750] - exit on exception args only applies to executors (#5169) (AlaeddineAbdessalem)
    • [55188165] - pin docarray version for new column syntax (#5171) (Joan Fontanals)
    • [fa83955c] - increase minimum protobuf version (#5166) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [28aeac8e] - add section for exit_on_exceptions argument (#5172) (Girish Chandrashekar)
    • [33891c46] - jcloud: labels in flow yaml (#5164) (Deepankar Mahapatro)

    ๐Ÿ Unit Test and CICD

    • [b59d0450] - add jina auth token (#5167) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [76dca020] - update announcement in readme and docs (Jina Dev Bot)
    • [f4248ea8] - docs: update TOC (Jina Dev Bot)
    • [fda5623a] - version: the next version will be 3.9.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.1(Sep 8, 2022)

    Release Note (3.9.1)

    Release time: 2022-09-08 14:10:44

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [273fda5a] - merge dryrun into ping (#5151) (Han Xiao)

    ๐Ÿน Other Improvements

    • [0a71009b] - version: the next version will be 3.9.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.0(Sep 8, 2022)

    Highlights :star2:

    Update to new protobuf version

    We have updated our protobuf dependency to use the latest protobuf version leveraging all the performance improvement that comes with it.

    #5082

    Merge manifest.yml and config.yml for Jina Hub Executors

    We have made it easier for the users to manage their Hub Executors by merging the information from manifest.yml to config.yml which makes the experience much better.

    #5101

    Expose grpc server parameters to Executor

    We expose all the parameters that grpc server can accept for users to be able to configure their services in the most robust and reliable way.

    #5092

    Other changes

    • Add more Dockerfile templates options in jina hub new #5104
    • Add failed and successfull requests metrics #5079

    Bug fixes

    • Fix reading empty list from environment variables #5143
    Source code(tar.gz)
    Source code(zip)
  • v3.8.4(Sep 8, 2022)

    Release Note (3.8.4)

    Release time: 2022-09-08 08:51:32

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, AlaeddineAbdessalem, Joan Fontanals, Johannes Messner, Andrei Ungureanu, samsja, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [d1faf6e6] - pass internal flag to telemetry (#5134) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [e501ddfd] - version: adapt to docker hub v2 (#5146) (Deepankar Mahapatro)
    • [d76e69ac] - fix parsing empty list (#5143) (AlaeddineAbdessalem)
    • [3ec33c08] - gpu dockerfile template include now jina install (#5124) (samsja)
    • [e2967f98] - set minimal docarray dependency (#5133) (Joan Fontanals)
    • [ff803686] - linkerd cd (#5131) (AlaeddineAbdessalem)

    ๐Ÿงผ Code Refactoring

    • [a78612f3] - manifest config (#5101) (Andrei Ungureanu)

    ๐Ÿ Unit Test and CICD

    • [ddf90efb] - remove outdated test (#5139) (Johannes Messner)
    • [a4ea7215] - increase wait time (#5137) (AlaeddineAbdessalem)
    • [478e53f6] - fix k8s tests return responses params (#5127) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a9b80aec] - docs: update TOC (Jina Dev Bot)
    • [1df9b459] - version: the next version will be 3.8.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.3(Sep 5, 2022)

    Release Note (3.8.3)

    Release time: 2022-09-05 14:48:30

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [7b7eecc5] - cap grpc version (#5122) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [47e2cfc3] - docs: update TOC (Jina Dev Bot)
    • [0dbdcaa0] - version: the next version will be 3.8.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.2(Sep 3, 2022)

    Release Note (3.8.2)

    Release time: 2022-09-03 15:56:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Zac Li, samsja, AlaeddineAbdessalem, Jina Dev Bot, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [e794c06c] - expose grpc parameters and add production ready keepalive parameters (#5092) (samsja)
    • [6205ffc9] - add gpu dockerfile support to jina hub new (#5104) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [9293698b] - rewrite flow add (Han Xiao)
    • [115bc0fa] - rewrite flow add (#5120) (Han Xiao)
    • [22fdbcee] - jcloud: fix jc status link (#5116) (Zac Li)
    • [4dfe6dfb] - jcloud: fix docs for gateway resource customization (#5114) (Zac Li)
    • [6b24bf69] - correct jcloud retention days (#5109) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [587f6268] - fix test concurrent async flow (#5115) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [763f6773] - update readme (Han Xiao)
    • [2651c506] - disable hub-integration (Han Xiao)
    • [3590c385] - fix readme (Han Xiao)
    • [8303fb25] - docs: improve docs on jcloud k8s (Han Xiao)
    • [a34edc62] - update announcement in readme and docs (Jina Dev Bot)
    • [d124d9af] - docs: update TOC (Jina Dev Bot)
    • [0df7cd8a] - version: the next version will be 3.8.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.1(Aug 30, 2022)

    Release Note (3.8.1)

    Release time: 2022-08-30 19:32:52

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6df36ce5] - client: add profiling function to client and flow (#5105) (Han Xiao)

    ๐Ÿž Bug fixes

    • [78aae025] - fix inconsistent docs and tests on py_modules (#5108) (Han Xiao)
    • [a725010d] - remove flakyness of monitoring integration tests in CI (#5106) (samsja)

    ๐Ÿงผ Code Refactoring

    • [90fa81f3] - move port monitoring to pods parsing (#5100) (samsja)

    ๐Ÿ“— Documentation

    • [95ee283c] - fix naming (#5102) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [d528d947] - fix cd pipeline (Han Xiao)
    • [57625350] - docs: update TOC (Jina Dev Bot)
    • [1ad06216] - docs: fix retention days description (Han Xiao)
    • [9f5f5879] - version: the next version will be 3.8.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.0(Aug 29, 2022)

    Release Note (3.8.0)

    Release time: 2022-08-29 16:43:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, tarrantro, Han Xiao, AlaeddineAbdessalem, samsja, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [527beb85] - bump protobuf version (#5082) (AlaeddineAbdessalem)
    • [c47cb716] - add failed and successful request number metrics (#5079) (AlaeddineAbdessalem)

    ๐Ÿž Bug fixes

    • [c81252ac] - update to protobuf 4.21 new types (#5098) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [6e51e815] - refactor data request handler monitoring (#5088) (samsja)

    ๐Ÿ“— Documentation

    • [b3fdb49c] - correct jcloud retention days (#5096) (tarrantro)
    • [98fb71f1] - jcloud: adjust jcloud monitoring docs(#5081) (Zac Li)
    • [dbea43f3] - remove kong/alb from jcloud document (#5080) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [27ec3b41] - cleanup pip install (#5094) (AlaeddineAbdessalem)
    • [c0c3bf9a] - downgrade linkerd version in CI (#5090) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [142c0bb1] - docs: add streamline value props (Han Xiao)
    • [af7f6763] - fix readme (Han Xiao)
    • [c39714fa] - docs: add install apple silicon (Han Xiao)
    • [5375b4d3] - update announcement in readme and docs (Jina Dev Bot)
    • [53553b14] - add security report (Han Xiao)
    • [cfe5dd70] - update readme (Han Xiao)
    • [e106796c] - docs: update furo deps (Han Xiao)
    • [079d04b0] - docs: add jcloud to first app (Han Xiao)
    • [64de8608] - docs: add comparing alternatives (Han Xiao)
    • [f9c2d8de] - docs: update TOC (Jina Dev Bot)
    • [2154f98f] - version: the next version will be 3.7.15 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.14(Aug 22, 2022)

    Release Note (3.7.14)

    Release time: 2022-08-22 13:52:20

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿน Other Improvements

    • [4f380c7e] - fix pprint (Han Xiao)
    • [0bd289fe] - docs: update TOC (Jina Dev Bot)
    • [96489914] - version: the next version will be 3.7.14 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.13(Aug 21, 2022)

    Release Note (3.7.13)

    Release time: 2022-08-21 21:03:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b7bf0ee1] - set workspace in executor init method (#5072) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [f6dcde38] - telemetry (#5078) (Han Xiao)

    ๐Ÿน Other Improvements

    • [8ad0ead5] - docs: refactor create project docs (Han Xiao)
    • [c7e19841] - update readme (Han Xiao)
    • [7b034d34] - docs: refactor telemetry docs (Han Xiao)
    • [76f48f8f] - docs: refactor jcloud docs (Han Xiao)
    • [e506da44] - docs: refactor hub docs (Han Xiao)
    • [b91f2084] - docs: refactor executors docs (Han Xiao)
    • [f8a63ccf] - docs: refactor flow basic docs (Han Xiao)
    • [885914ad] - docs: restructure docs (Han Xiao)
    • [36fdebe2] - docs: fix docs in jcloud (Han Xiao)
    • [4673e7ab] - docs: update TOC (Jina Dev Bot)
    • [a4c48d1b] - version: the next version will be 3.7.13 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.12(Aug 18, 2022)

    Release Note (3.7.12)

    Release time: 2022-08-18 22:23:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6f87524c] - include jcloud cli (#5074) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [10c63db2] - update pr template with reminder to add a description (#5062) (Johannes Messner)
    • [6fbd2be9] - docs: update TOC (Jina Dev Bot)
    • [7d82fdc3] - version: the next version will be 3.7.12 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.11(Aug 17, 2022)

    Release Note (3.7.11)

    Release time: 2022-08-17 21:09:48

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [15fd60d5] - cli: jina auth cli error (Han Xiao)

    ๐Ÿน Other Improvements

    • [80393316] - version: the next version will be 3.7.11 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.10(Aug 17, 2022)

    Release Note (3.7.10)

    Release time: 2022-08-17 12:22:49

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b9bddd22] - cli: jina auth cli error (#5070) (Han Xiao)

    ๐Ÿ“— Documentation

    • [840c221f] - add gateway timeout and update shared gpu (#5069) (tarrantro)

    ๐Ÿน Other Improvements

    • [bffe263c] - docs: update TOC (Jina Dev Bot)
    • [94c83cdb] - version: the next version will be 3.7.10 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.9(Aug 16, 2022)

    Release Note (3.7.9)

    Release time: 2022-08-16 16:25:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    ๐Ÿงผ Code Refactoring

    • [c1c500aa] - remove unused hub environment variables (#5067) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [442a86cd] - version: the next version will be 3.7.9 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.8(Aug 16, 2022)

    Release Note (3.7.8)

    Release time: 2022-08-16 15:18:50

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [bf864748] - hub: use hubble sdk (#5064) (Han Xiao)

    ๐Ÿ“— Documentation

    • [317e9ce1] - jcloud: doc for gateway customization (#5065) (Zac Li)

    ๐Ÿน Other Improvements

    • [a6be9481] - update announcement in readme and docs (Jina Dev Bot)
    • [f3bcdaa8] - docs: update TOC (Jina Dev Bot)
    • [dfd1e07d] - docs: fix typos (Han Xiao)
    • [d579cb63] - version: the next version will be 3.7.8 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.7(Aug 15, 2022)

    Release Note (3.7.7)

    Release time: 2022-08-15 18:14:16

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [22563074] - cache hub pull with build env (#5061) (Johannes Messner)

    ๐Ÿน Other Improvements

    • [e7377947] - docs: fix typos (Han Xiao)
    • [6d218fbf] - version: the next version will be 3.7.7 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.6(Aug 15, 2022)

    Release Note (3.7.6)

    Release time: 2022-08-15 14:04:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [49a842d9] - unify temp path to dot cache jina (#5058) (Han Xiao)

    ๐Ÿ“— Documentation

    • [731d492c] - add CPU and GPU type (#5060) (tarrantro)
    • [d8f6f418] - rewrite the telemetry session (#5059) (Han Xiao)
    • [01111539] - jcloud: autoscale docs (#5056) (Zac Li)

    ๐Ÿน Other Improvements

    • [299a908b] - docs: add what is jina (Han Xiao)
    • [23914a81] - add what is cross multi modal (Han Xiao)
    • [0c78b993] - docs: update TOC (Jina Dev Bot)
    • [102bc6de] - version: the next version will be 3.7.6 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.5(Aug 10, 2022)

    Release Note (3.7.5)

    Release time: 2022-08-10 10:19:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Jina Dev Bot, zhangkai, samsja, Deepankar Mahapatro, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6cc4497a] - hubble support env variables (#5023) (zhangkai)
    • [12404535] - add dryrun to cli (#5050) (samsja)

    ๐Ÿ“— Documentation

    • [1d60655d] - jcloud: fix formatting (#5048) (Deepankar Mahapatro)
    • [6df83cf1] - improve grpc third party client documentation (#5047) (samsja)
    • [8c6321ac] - refactor resources, add gpu, kong (#5027) (tarrantro)

    ๐Ÿน Other Improvements

    • [d6d4c30a] - update announcement in readme and docs (Jina Dev Bot)
    • [9fee2dff] - docs: update TOC (Jina Dev Bot)
    • [fe7b8894] - version: the next version will be 3.7.5 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.4(Aug 3, 2022)

    Release Note (3.7.4)

    Release time: 2022-08-03 12:31:57

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [4c3d760f] - do not send target_executor to Executors (#5041) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a51b42cc] - update announcement in readme and docs (Jina Dev Bot)
    • [ecb95829] - docs: update TOC (Jina Dev Bot)
    • [286bdce8] - version: the next version will be 3.7.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.3(Jul 28, 2022)

    Release Note (3.7.3)

    Release time: 2022-07-28 14:29:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [032bd5e6] - fix specific params problem with branches (#5038) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [81999e97] - version: the next version will be 3.7.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.2(Jul 28, 2022)

    Release Note (3.7.2)

    Release time: 2022-07-28 09:19:06

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [fb88015e] - add more logging in grpc dry run (#5036) (Zac Li)

    ๐Ÿน Other Improvements

    • [5116b922] - docs: update TOC (Jina Dev Bot)
    • [32dd47af] - version: the next version will be 3.7.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

null 139 Jan 1, 2023
A python software that can help blind people find things like laptops, phones, etc the same way a guide dog guides a blind person in finding his way.

GuidEye A python software that can help blind people find things like laptops, phones, etc the same way a guide dog guides a blind person in finding h

Munal Jain 0 Aug 9, 2022
Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

Jamin Fong 291 Nov 18, 2022
Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

null 75 Nov 24, 2022
MPRNet-Cloud-removal: Progressive cloud removal

MPRNet-Cloud-removal Progressive cloud removal Requirements 1.Pytorch >= 1.0 2.Python 3 3.NVIDIA GPU + CUDA 9.0 4.Tensorboard Installation 1.Clone the

Semi 95 Dec 18, 2022
This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

The Neural Process Family This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CN

DeepMind 892 Dec 28, 2022
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

DeepHyper Team 214 Jan 8, 2023
This is the paddle code for SeBoW(Self-Born wiring for neural trees), a kind of neural tree born form a large search space

SeBoW: Self-Born Wiring for neural trees(PaddlePaddle version) This is the paddle code for SeBoW(Self-Born wiring for neural trees), a kind of neural

HollyLee 13 Dec 8, 2022
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022
This repo contains the code and data used in the paper "Wizard of Search Engine: Access to Information Through Conversations with Search Engines"

Wizard of Search Engine: Access to Information Through Conversations with Search Engines by Pengjie Ren, Zhongkun Liu, Xiaomeng Song, Hongtao Tian, Zh

null 19 Oct 27, 2022
Deep Text Search is an AI-powered multilingual text search and recommendation engine with state-of-the-art transformer-based multilingual text embedding (50+ languages).

Deep Text Search - AI Based Text Search & Recommendation System Deep Text Search is an AI-powered multilingual text search and recommendation engine w

null 19 Sep 29, 2022
GndNet: Fast ground plane estimation and point cloud segmentation for autonomous vehicles using deep neural networks.

GndNet: Fast Ground plane Estimation and Point Cloud Segmentation for Autonomous Vehicles. Authors: Anshul Paigwar, Ozgur Erkent, David Sierra Gonzale

Anshul Paigwar 114 Dec 29, 2022
Lightweight library to build and train neural networks in Theano

Lasagne Lasagne is a lightweight library to build and train neural networks in Theano. Its main features are: Supports feed-forward networks such as C

Lasagne 3.8k Dec 29, 2022
Lightweight library to build and train neural networks in Theano

Lasagne Lasagne is a lightweight library to build and train neural networks in Theano. Its main features are: Supports feed-forward networks such as C

Lasagne 3.8k Feb 11, 2021
HeatNet is a python package that provides tools to build, train and evaluate neural networks designed to predict extreme heat wave events globally on daily to subseasonal timescales.

HeatNet HeatNet is a python package that provides tools to build, train and evaluate neural networks designed to predict extreme heat wave events glob

Google Research 6 Jul 7, 2022
Neural network-based build time estimation for additive manufacturing

Neural network-based build time estimation for additive manufacturing Oh, Y., Sharp, M., Sprock, T., & Kwon, S. (2021). Neural network-based build tim

Yosep 1 Nov 15, 2021
Build upon neural radiance fields to create a scene-specific implicit 3D semantic representation, Semantic-NeRF

Semantic-NeRF: Semantic Neural Radiance Fields Project Page | Video | Paper | Data In-Place Scene Labelling and Understanding with Implicit Scene Repr

Shuaifeng Zhi 243 Jan 7, 2023
A series of convenience functions to make basic image processing operations such as translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with OpenCV and Python.

imutils A series of convenience functions to make basic image processing functions such as translation, rotation, resizing, skeletonization, and displ

Adrian Rosebrock 4.3k Jan 8, 2023