Release Note (3.13.2)

This release contains 1 bug fix.

🐞 Bug Fixes

Respect timeout_ready when generating startup probe (#5560)

As Kubernetes Startup Probes were added to all deployments in release v3.13.0, we added default values for all probe configurations. However, if those default configurations were not enough to wait for an Executor that takes time to load and become ready, the Executor deployment would become subject to the configured restart policy. Therefore, Executors that are slow to load would keep restarting forever.

In this patch, this behavior is fixed by making sure that Startup Probe configurations respect the timeout_ready argument of Executors. Startup Probes are configured like so:

• periodSeconds always set to 5 seconds
• timeoutSeconds always set to 10 seconds
• failureThreshold is the number of attempts maybe by kubernetes to check if the pod is ready. It varies according to timeout_ready. The formula used is failureThreshold = timeout_ready / 5000 (as timeout_ready is in microseconds and periodSeconds is 5 seconds) and in all cases, it will be at least 3. If timeout_ready is -1 (in Jina it means waiting forever for the Executor to become ready), since waiting forever is not supported in Kubernetes, the value for failureThreshold will be 120 attempts.

🤘 Contributors

We would like to thank all contributors to this release:

• AlaeddineAbdessalem (@alaeddine-13)

Release Note (3.13.1)

Release time: 2022-12-21 12:58:15

This release contains 3 bug fixes and 1 documentation improvement.

🐞 Bug Fixes

Support Gateway with multiple protocols for Kubernetes export (#5532)

You can now export Flows with multiple protocols to Kubernetes. Previously this would cause an error.

flow = Flow().config_gateway(protocol=['http', 'grpc'])
flow.to_kubernetes_yaml('k8s_flow_folder')

Fix Python 3.11 support (#5529)

It was previously impossible to install Jina with Python 3.11 due to a grpcio dependency problem. grpcio added support for Python 3.11 only with version 1.49.0, causing potential problems when used by Jina and other projects.

In this release grpcio>=1.49.0 is installed alongside Jina when using Python 3.11. However, be aware of potential problems related to grpc hanging.

Unary RPC from Client respects results_in_order (#5513)

In prior releases, calling the post method of a client with grpc and using stream=False did not respect the results_in_order parameter and results were always returned in order:

# this wrongly returns results in order
c = Client(protocol='grpc')
c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)  

Also this implied that using the Client with asyncio=True and stream=False in the post call would return results in the order that they were returned by the Flow, rather than respecting the input order:

# this wrongly returns results in order
c = Client(protocol='grpc', asyncio=True)

async for resp in c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)
    print(resp)

This release fixes the ordering bug.

📗 Documentation Improvements

• Document inheritance of arguments from Flow API to Executors and Gateway (#5535)

🤘 Contributors

We would like to thank all contributors to this release:

• AlaeddineAbdessalem (@alaeddine-13)
• Joan Fontanals (@JoanFM)
• Jackmin801 (@Jackmin801)
• Anne Yang (@AnneYang720)

Release Note (3.13.0)

Release time: 2022-12-15 15:33:43

This release contains 14 new features, 9 bug fixes and 7 documentation improvements.

This release introduces major features like Custom Gateways, Dynamic Batching for Executors, development support with auto-reloading, support for the new namespaced Executor scheme jinaai, improvements for our gRPC transport layer, and more.

🆕 Features

Custom Gateways (#5153, #5189, #5342, #5457, #5465, #5472 and #5477)

Jina Gateways are now customizable in the sense that you can implement them in much the same way as an Executor. With this feature, Jina gives power to the user to implement any server, protocol or interface at the Gateway level. There's no more need to build an extra service that uses the Flow.

For instance, you can define a Jina Gateway that communicates with the rest of Flow Executors like so:

from docarray import Document, DocumentArray
from jina.serve.runtimes.gateway.http.fastapi import FastAPIBaseGateway


class MyGateway(FastAPIBaseGateway):
    @property
    def app(self):
        from fastapi import FastAPI

        app = FastAPI(title='Custom FastAPI Gateway')

        @app.get(path='/service')
        async def my_service(input: str):
            # convert input request to Documents
            docs = DocumentArray([Document(text=input)])

            # send Documents to Executors using GatewayStreamer
            result = None
            async for response_docs in self.streamer.stream_docs(
                docs=docs,
                exec_endpoint='/',
            ):
                # convert response docs to server response and return it
                result = response_docs[0].text

            return {'result': result}

        return app

Then you can use it in your Flow in the following way:

flow = Flow().config_gateway(uses=MyGateway, port=12345, protocol='http')

A Custom Gateway can be used as a Python class, YAML configuration or Docker image.

Adding support for Custom Gateways required exposing the Gateway API and supporting multiple ports and protocols (mentioned in a prior release). You can customize it by subclassing the FastAPIBaseGateway class (for simple implementation) or base Gateway for more complex use cases.

Working on this feature also involved exposing and improving the GatewayStreamer API as a way to communicate with Executors within the Gateway.

Find more information in the Custom Gateway page.

Dynamic batching (#5410)

This release adds Dynamic batching capabilities to Executors.

Dynamic batching allows requests to be accumulated and batched together before being sent to an Executor. The batch is created dynamically depending on the configuration for each endpoint.

This feature is especially relevant for inference tasks where model inference is more optimized when batched to efficiently use GPU resources.

You can configure Dynamic batching using either a decorator or the uses_dynamic_batching parameter. The following example shows how to enable Dynamic batching on an Executor that performs model inference:

from jina import Executor, requests, dynamic_batching, Flow, DocumentArray, Document
import numpy as np
import torch


class MyExecutor(Executor):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)

        # initialize model
        self.model = torch.nn.Linear(in_features=128, out_features=128)

    @requests(on='/bar')
    @dynamic_batching(preferred_batch_size=10, timeout=200)
    def embed(self, docs: DocumentArray, **kwargs):
        docs.embeddings = self.model(torch.Tensor(docs.tensors))


flow = Flow().add(uses=MyExecutor)

With Dynamic Batching enabled, the Executor above will efficiently use GPU resources to perform inference by batching Documents together.

Read more about the feature in the Dynamic Batching documentation page.

Install requirements of local Executors (#5508)

Prior to this release, the install_requirements parameter of Executors only installed Executor requirements for Hub Executors. Now, local Executors with a requirements.txt file will also have their requirements installed before starting Flows.

Support jinaai Executor scheme to enable namespaced Hub Executors (#5462, #5468 and #5515)

As Jina AI Cloud introduced namespaces to Executor resources, we made changes to support the new jinaai Executor scheme. This PR adds support for the new scheme.

This means that namespaced Executors can now be used with the jinaai scheme in the following way:

from jina import Flow

flow = Flow().add(uses='jinaai://jina-ai/DummyHubExecutor')

This scheme is also supported in Kubernetes and other APIs:

from jina import Flow

flow = Flow().add(uses='jinaai+docker://jina-ai/DummyHubExecutor')
flow.to_kubernetes_yaml('output_path', k8s_namespace='my-namespace')

The support of the new scheme means the minimum supported version of jina-hubble-sdk has been increased to 0.26.10.

Add auto-reloading to Flow and Executor on file changes (#5461, #5488 and #5514)

A new argument reload has been added to the Flow and Executor APIs, which automatically reloads running Flows and Executors when changes are made to Executor source code or YAML configurations of Flows and Executors.

Although this feature is only meant for development, it aims to help developers iterate fast and automatically update Flows with changes they make live during development.

Find out more about this feature in these two sections:

• Executor hot reload
• Flow hot reload

Expand Executor serve parameters (#5494)

The method Executor.serve can receive more parameters, similar to what the Flow API expects. With new parameters to control serving and deployment configurations of the Executor, this method empowers the Executor to be convenient for single service tasks.

This means you can not only build advanced microservices-based pipelines and applications, but also build individual services with all Jina features: shards/replicas, dynamic batching, auto-reload, etc.

Read more about the method in the Python API documentation.

Add gRPC trailing metadata when logging gRPC error (#5512)

When logging gRPC errors, context trailing metadata is now shown. This helps identify underlying network issues rather than the error codes that mask multiple network errors into a single gRPC status code.

For instance, the new log message looks like the following:

DEBUG  gateway@ 1 GRPC call to deployment executor0 failed                      
       with error <AioRpcError of RPC that terminated with:                     
               status = StatusCode.UNAVAILABLE                                  
               ...                                                                   
       trailing_metadata=Metadata((('content-length', '0'),                     
       ('l5d-proxy-error', 'HTTP Balancer service in                            
       fail-fast'), ('l5d-proxy-connection', 'close'),                          
       ('date', 'Tue, 13 Dec 2022 10:20:15 GMT'))), for                         
       retry attempt 2/3. Trying next replica, if available. 

The trailing_metadata returned by load balancers will help to identify the root cause more accurately.

Implement unary_unary stub for Gateway Runtime (#5507)

This release adds the gRPC unary_unary stub for Gateway Runtime as a new communication stub with Executors. Since the gRPC performance best practices for Python page suggests that unary stream implementation might be faster for Python, we added this communication method.

However, this is not enabled by default. The streaming RPC method will still be used unless you set the stream option to False in the Client.post() method. The feature is only effective when the gRPC protocol is used.

Read more about the feature in the documentation: https://docs.jina.ai/concepts/client/send-receive-data/#use-unary-or-streaming-grpc

Add Startup Probe and replace Readiness Probe with Liveness Probe (#5407)

Before this release, when exporting Jina Flows to Kubernetes YAML configurations, Kubernetes Readiness Probes used to be added for the Gateway pod and each Executor pod. In this release we have added a Startup Probe and replaced Readiness Probe with Liveness Probe.

Both probes use the jina ping command to check that pods are healthy.

New Jina perf Docker images (#5498)

We added a slightly larger Docker image with suffix perf which includes a set of tools useful for performance tuning and debugging.

The new image is available in Jina AI's Docker hub.

New Jina Docker image for Python 3.10, and use Python 3.8 for default Jina image (#5490)

Besides adding Docker images aimed for performance optimization, we added an image with a newer Python version: 3.10. This is available in Jina AI's Docker hub, for example jinaai/jina:master-py310-standard.

We also made Python 3.8 our minimum supported Python version by default, and it will be used for default Docker images.

Minimize output of jina ping command (#5476)

jina ping commands are now less verbose and will print less irrelevant output. However, important information like latency for each round, average latency, number of successful requests and ping result will still show up.

Add Kubernetes preStop hook to the containers (#5445)

A preStop hook has been added to Executors and the Gateway to allow a grace period. This allows more time to complete in-flight requests and finish the server's graceful shutdown.

Generate random ports for multiple protocols (#5455)

If you use multiple protocols for a Gateway, you no longer need to specify a port for each one. Whether it's Python or YAML, you just need to specify the protocols you want to support and Jina will generate random ports for you.

Python API:

from jina import Flow

flow = Flow().config_gateway(protocol=['grpc', 'http', 'websocket'])
with flow:
    flow.block()

YAML:

jtype: Flow
gateway:
  protocol:
    - 'grpc'
    - 'http'
    - 'websocket'

Result:

🐞 Bug Fixes

List-like args passed as string (#5464)

We fixed the format expected for port, host and port_monitoring to feel more Pythonic. Basically, if you use replicas, you no longer have to provide comma-separated ports as a string value. Instead, you can simply pass a list of values, no need to put all in a string anymore!

For instance, suppose we have two external replicas of an Executor that we want to join in our Flow (the first is hosted on localhost:12345 and the second on 91.198.174.192:12346). We can add them like this:

from jina import Flow

replica_hosts, replica_ports = ['localhost', '91.198.174.192'], [
    '12345',
    '12346',
]  # instead of 'localhost,91.198.174.192', '12345,12346'
Flow().add(host=replica_hosts, port=replica_ports, external=True)

Or:

Flow().add(host=['localhost:12345', '91.198.174.192:12346'], external=True)

Note that this is not a breaking change, and the old syntax (comma-separated values: Flow().add(host='localhost:12345,91.198.174.192:12346', external=True)) is still supported for backwards compatibility.

Restore port to overload type hint and JSON schema (#5501)

When we made port and protocol arguments of the Gateway support multiple values, a bug was introduced where port did not appear in Jina's JSON schema as well as the Flow API overload for method signatures.

Although the arguments are functional in both the Python API and YAML, this suppressed auto-completion and developer support for these parameters. This release restores the port parameter in both the Flow method overloads and JSON schema.

Do not force insecure to True in open telemetry integration (#5483)

In Jina's instrumentation, communication to open telemetry exporters used to be forced to insecure mode. Luckily, our community member @big-thousand picked this up and submitted a fix. The communication is no longer forced to the insecure mode.

Kudos to @big-thousand for his contribution!

Fix problem when using floating Executor in HTTP (#5493)

We found a bug when using Floating Executors in HTTP, where the floating Executor is connected to the Gateway (in the Flow topology). In this case, the Executor would not receive input Documents properly. This release fixes the mentioned bug.

Add egg info post install command for egg info setup mode (#5491)

This release adds support for the egg info setup mode in Python. This means post-installation commands are now properly executed in environments that rely on Python's new setup mode.

This bug resulted in several issues especially for environments that depend on these post-installation commands. For instance, some Environment Variables that are needed for Jina to work on macOS and for CLI auto-complete.

Do not apply limits when gpus='all' in Kubernetes (#5485)

If Executor parameter gpus is set to "all", no limits will be applied on the pod in Kubernetes.

Fix Windows signal handling (#5484)

This release improves signal handling on Windows, specifically when cancelling a Flow with an OS signal.

Cap opentelemetry-instrumentation-aiohttp-client (#5452)

This release caps the version for opentelemetry-instrumentation-aiohttp-client which is incompatible with opentelemetry-semantic-conventions.

Raise exceptions from path importer (#5447)

Previously, errors were hidden when they came from a Python module imported to load an Executor. Actually the module was not considered to be a Python module, which produced other misleading errors. In this release, actual errors during imports will be raised and no longer hidden.

📗 Documentation Improvements

• Add gRPC requirements for Apple Silicon (M1 Chip) to fix failing installation of Jina (#5511)
• Add redirects from '/fundamentals' to '/concepts' (#5504)
• Update JCloud documentation to the jcloud v0.1.0 (#5385)
• Restructure documentation under /concepts
• Change Executor URI scheme to namespaced scheme jinaai (#5450)
• Custom Gateway documentation (#5465)
• Provide more accurate description for port and protocol parameters of the Gateway (#5456)

🤟 Contributors

We would like to thank all contributors to this release:

• Delgermurun (@delgermurun)
• Jie Fu (@jemmyshin)
• Alex Cureton-Griffiths (@alexcg1)
• big-thousand (@big-thousand)
• IyadhKhalfallah (@IyadhKhalfallah)
• Deepankar Mahapatro (@deepankarm)
• samsja (@samsja)
• AlaeddineAbdessalem (@alaeddine-13)
• Joan Fontanals (@JoanFM)
• Anne Yang (@AnneYang720)
• Han Xiao (@hanxiao)
• Girish Chandrashekar (@girishc13)
• Jackmin801 (@Jackmin801)

Release Note (3.12.0)

Release time: 2022-11-25 12:28:29

This release contains 8 new features, 16 bug fixes and 15 documentation improvements.

🆕 Features

Support multiple protocols at the same time in Flow Gateways (#5435 and #5378)

Prior to this release, a Flow only exposed one server in its Gateway with one of the following protocols: HTTP, gRPC or WebSockets.

Now, you can specify multiple protocols and for each one, a separate server is started. Each server is bound to its own port.

For instance, you can do:

from jina import Flow
flow = Flow(port=[12345, 12346, 12347], protocol=['http', 'grpc', 'websocket'])
with flow:
    flow.block()

or: jina flow --uses flow.yml where flow.yml is:

jtype: Flow
with:
  protocol:
    - 'grpc'
    - 'http'
    - 'websocket'
  port:
    - 12345
    - 12344
    - 12343

The protocol and port parameters can still accept single values rather than a list. Therefore, there is no breaking change.

Alias parameters protocols and ports are also defined:

flow = Flow(ports=[12345, 12346, 12347], protocols=['http', 'grpc', 'websocket'])

In Kubernetes, this exposes separate services for each protocol.

Read the docs for more information.

Add Kubernetes information to resource attributes in instrumentation (#5372)

When deploying to Kubernetes, the Gateway and Executors expose the following Kubernetes information as resource attributes in instrumentation:

• k8s.namespace.name
• k8s.pod.name
• k8s.deployment.name / k8s.statefulset.name

Besides that, the following resource attributes are set if they are present in the environment variables of the container:

• k8s.cluster.name (set K8S_CLUSTER_NAME environment variable)
• k8s.node.name (set K8S_NODE_NAME environment variable)

Add option to return requests in order using the Client (#5404)

If you use replicated Executors, those which finish processing first return their results to the Gateway which then returns them to the client. This is useful if you want results as soon as each replicated Executor finishes processing your Documents.

However, this may be inconvenient if you want the Documents you send to the Flow to return in order. In this release, you can retain the order of sent Documents (when using replicated Executors) by passing the results_in_order parameter in the Client.

For instance, if your Flow looks like this:

from jina import Flow, DocumentArray, Document
f = Flow().add(replicas=2)

You can do the following to keep results in order:

input_da = DocumentArray([Document(text=f'{i}') for i in range(100)])

with f:
    result_da = f.post('/', inputs=input_da, request_size=10,  results_in_order=True)
    assert result_da[:, 'text'] == input_da[:, 'text']

Add docs_map parameter to Executor endpoints (#5366)

Executor endpoint signatures are extended to the following:

class MyExecutor(Executor):
    @requests
    async def foo(
        self, docs: DocumentArray, parameters: Dict, docs_matrix: Optional[List[DocumentArray]], docs_map: Optional[Dict[str, DocumentArray]]
    ) -> Union[DocumentArray, Dict, None]:
        pass

Basically, the parameter docs_map has been added. It's a dictionary that maps previous Executor names to DocumentArrays. This is useful when you have an Executor that combines results from many previous Executors, and you need information about where each resulting DocumentArray comes from.

Add Gateway API (#5342)

Prior to this release, all Gateway configurations were specified in the Flow API. However, by principle, Flow parameters are commonly inherited by Executors and the Gateway. We already gave the Executor its own API to be customized (either using the method add() or the executors YAML section in Flow YAML).

In this release, we have done the same for Gateway. It defines its own API in both the Python API and YAML interface. In the Python API, you can configure the Gateway using the config_gateway() method:

flow = Flow().config_gateway(port=12345, protocol='http')

And in the YAML interface, you can configure the Gateway using the gateway section:

!Flow
gateway:
  protocol: http
  port: 12344
executors:
  - name: exec

This is useful when you want to apply parameters just for the Gateway. If you want a parameter to be applied to all Executors, then continue to use the Flow API.

Keep in mind that you can still provide Gateway parameters using the Flow API. This means there are no breaking changes introduced.

Support UUID in CUDA_VISIBLE_DEVICES round-robin assignment (#5360)

You can specify a comma-separated list of GPU UUIDs in the CUDA_VISIBLE_DEVICES to assign devices to Executor replicas in a round-robin fashion. For instance:

CUDA_VISIBLE_DEVICES=RRGPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5,GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5,GPU-0ccccccc-74d2-7297-d557-12771b6a79d5,GPU-0ddddddd-74d2-7297-d557-12771b6a79d5

Check CUDA's documentation to see the accepted formats to assign CUDA devices by UUID.

| GPU device | Replica ID | |------------------------------------------|------------| | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 0 | | GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5 | 1 | | GPU-0ccccccc-74d2-7297-d557-12771b6a79d5 | 2 | | GPU-0ddddddd-74d2-7297-d557-12771b6a79d5 | 3 | | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 4 |

Thanks to our community member @mchaker for submitting this feature request!

Capture shard failures in the head runtime (#5338)

In case you use Executor shards, partially failed requests (those that fail on a subset of the shards) no longer raise an error.

Instead, successful results are returned. An error is raised only when all shards fail to process Documents. Basically, the HeadRuntime's behavior is updated to fail only when all shards fail.

Thanks to our community user @soumil1 for submitting this feature request.

Add successful, pending and failed metrics to HeadRuntime (#5374)

More metrics have been added to the Head Pods:

• jina_number_of_pending_requests: number of pending requests
• jina_successful_requests: number of successful requests
• jina_failed_requests: number of failed requests
• jina_received_request_bytes: the size of received requests in bytes
• jina_sent_response_bytes: the size of sent responses in bytes

See more in the instrumentation docs.

Add deployment label in grpc stub metrics (#5344)

Executor metrics used to show up aggregated at the Gateway level and users couldn't see separate metrics per Executor. With this release, we have added labels for Executors so that metrics in the Gateway can be generated per Executor or aggregated over all Executors.

🐞 Bug Fixes

Check whether the deployment is in Executor endpoints mapping (#5440)

This release adds an extra check in the Gateway when sending requests to deployments: The Gateway sends requests to the deployment only if it is in the Executor endpoint mapping.

Unblock event loop to allow health service (#5433)

Prior to this release, sync function calls inside Executor endpoints blocked the event loop. This meant that health-checks submitted to Executors failed for long tasks (for instance, inference using a large model).

In this release, such tasks no longer block the event loop. While concurrent requests to the same Executor wait until the sync task finishes, other runtime tasks remain functional, mainly health-checks.

Dump environment variables to string for Kubernetes (#5430)

Environment variables are now cast to strings before dumping them to Kubernetes YAML.

Unpin jina-hubble-sdk version (#5412)

This release frees (unpins) jina-hubble-sdk version. The latest jina-hubble-sdk is installed with the latest Jina.

Bind servers to host argument instead of __default_host__ (#5405)

This release makes servers at each Jina pod (head, Gateway, worker) bind to the host address specified by the user, instead of always binding to the __default_host__ corresponding to the OS. This lets you, depending on your network interface, restrict or expose your Flow services in your network.

For instance, if you wish to expose all pods to the internet, except for the last Executor, you can do:

flow = Flow(host='0.0.0.0').add().add(host='127.0.0.1')

After this fix, Jina respects this syntax and binds the last Executor only to 127.0.0.1 (accessible only inside the host machine).

Thanks to @wqh17101 for reporting this issue!

Fix backoff_multiplier format when using max_attempts in the Client (#5403)

This release fixes the format of backoff_multiplier parameter when injected into the gRPC request. The issue appeared when you use the max_attempts parameter in Client.

Maintain the correct tracing operations chain (#5391)

Tracing spans for Executors used to show up out of order. This behavior has been fixed by using the method start_as_current_span instead of start_span to maintain the tracing chain in the correct order.

Use Async health servicer for tracing interceptors when tracing is enabled (#5392)

When tracing is enabled, health checks in Docker and Kubernetes deployments used to fail silently until the Flow timed out. This happened because tracing interceptors expected RPC stubs to be coroutines.

This release fixes this issue by using the async aio.HealthServicer instead of grpc_health.HealthServicer. Health checks submitted to runtimes (Gateway, head, worker) no longer fail when tracing is enabled.

Properly update requests count in case of Exception inside the HeadRuntime (#5383)

In case of an Exception being raised in the HeadRuntime, request counts were not updated properly (pending requests should have been decremented and failed requests should have been incremented). This is fixed in this release and the Exception is caught to update request counts.

Fix endpoint binding when inheriting Executors (#5380)

When an Executor is inherited, the bound endpoints of the parent Executor used to be overridden by those of the child Executor. This meant, if you inherited from Executors but still chose to use the parent Executor in your Flow, a wrong endpoint could have been called. This behavior is fixed by making Executor.requests a nested dict that also includes information about the class name. This helps to properly support Executor inheritance.

Missing recording logic in connection stub metrics (#5363)

Recording of request and response size in bytes is fixed to track all cases. This makes these metrics more accurate for the Gateway.

Move build configs to pyproject (#5351)

Build requirements have been moved from setup.py to pyproject.toml. This suppresses deprecation warnings that show up when installing Jina.

New timer should keep labels (#5341)

The MetricsTimer in instrumentation previously created new timers without keeping the histogram metric labels. This behavior is fixed and new timers retain the same labels.

Use non-mutable default for MetricsTimer constructor (#5339)

Use None instead of empty dict as a default value for histogram_metric_labels in MetricsTimer constructor.

Catch RpcErrors and show better error messages in Client (#5325)

In the Client, we catch RpcError and show its details instead of showing a standard error message.

Import OpenTelemetry functions only when tracing is enabled in WorkerRuntime (#5321)

This release ensures OpenTelemetry functions are only imported when tracing is enabled in the worker.

📗 Documentation Improvements

• Remove 3 off-topic articles
• Enable flag to convert resource labels to metric labels (#5409)
• Add reference to Jina Kotlin client from community (#5390)
• Add section about the transition from DocArray to Jina (#5382)
• Add tips about supporting multiprocessing with fork in Jina when using macOS (#5379)
• Fix reference to multiprocessing with spawn section and emphasize the need for entrypoint protection (#5370)
• Update instructions to build protos locally using protogen Docker image (#5335)
• Change mentions of JCloud to Jina AI Cloud (#5329)
• Restructure docs into cloud-native section (#5332)
• Add contributor acknowledgement and spell checking (#5324)
• Create a Kubernetes section (#5315)
• Add reference to Go and PHP clients (#5253)
• Introduce versioning to the documentation (#5310)
• Support redirects for removed documentation pages (#5301)
• Use better Grafana screenshot without random text block (#5306)

🤟 Contributors

We would like to thank all contributors to this release:

• Ziniu Yu (@ZiniuYu)
• Andrei Ungureanu (@Andrei997)
• Alex Cureton-Griffiths (@alexcg1)
• Yanlong Wang (@nomagick)
• Johannes Messner (@JohannesMessner)
• samsja (@samsja)
• Joan Fontanals (@JoanFM)
• Zhaofeng Miao (@mapleeit)
• Girish Chandrashekar (@girishc13)
• Jackmin801 (@Jackmin801)
• Han Xiao (@hanxiao)
• AlaeddineAbdessalem (@alaeddine-13)
• Nan Wang (@nan-wang)

Release note

This release contains 1 hot fix.

🐞 Bug Fix

Unpin jina-hubble-sdk

To avoid conflicting dependency versions with docarray, unpin the jina-hubble-sdk version dependency

🤟 Contributors

Joan Fontanals (@JoanFM )

Release Note (3.11.0)

Release time: 2022-10-24 14:42:35

This release contains 6 new features, 1 bug fix and 10 documentation improvements.

🆕 Features

Add OpenTelemetry tracing and metrics with basic configuration (#5175)

Jina now supports OpenTelemetry Tracing and Metrics libraries for increased observability and instrumentation of Jina Runtimes. Read the docs (#5291) for details. We also provide a migration guide for a smooth transition from the previously-supported Prometheus-only based metrics instrumentation.

You can integrate Jaeger or other distributed tracing tools to collect and visualize request-level and application level service operation attributes. This helps you analyze request-response lifecycle, application behavior and performance.

from jina import Executor, requests, DocumentArray

class MyExec(Executor):
    @requests
    def encode(self, docs: DocumentArray, **kwargs):
        with self.tracer.start_as_current_span(
            'encode', context=tracing_context
        ) as span:
            with self.monitor(
                'preprocessing_seconds', 'Time preprocessing the requests'
            ):
                docs.tensors = preprocessing(docs)
            with self.monitor(
                'model_inference_seconds', 'Time doing inference the requests'
            ):
                docs.embedding = model_inference(docs.tensors)

Record existing Prometheus metrics into OpenTelemetry histograms (#5275)

Prometheus-only based metrics are also available as OpenTelemetry supported metrics. You can switch to OpenTelemetry metrics with little effort.

Add default tracing interceptors to head gRPC connection pool (#5271)

Head Runtime supports default traces when forwarding requests to the shard replica.

Add gRPC metadata to Executors and Gateway deployments (#5221)

You can now provide HTTP header access tokens as client metadata for accessing external and authentication protected services.

from jina import Client

client = Client(host='api.clip.jina.ai', port=2096, tls=True, protocol='grpc')
client.post(on='/encode', metadata=(('authorization', '<your access token>'),))

Show local location of Executors in Hub (#5282)

Use the list sub command to list the locations of local Hub Executors:

jina hub list

Dump to Statefulset in K8s when volumes are passed to Executor (#5265)

Jina will generate a Kubernetes Statefulset manifest for Executors if you provide volumes to the Flow.to_kubernets_yaml() method. Executors can persist data on mounted volumes and correctly scale horizontally if required.

🐞 Bug Fixes

Invalid input raises exception (#5141)

• Theclient.post() method was not raising an exception if any of the underlying async requests raised an exception.
• The new version correctly raises the exception raised by any async request. The user can now correctly catch any exceptions and act upon the exception.

📗 Documentation Improvements

• Docs for OpenTelemetry instrumentation (#5291)
• Add Jina AI Cloud restructure Jcloud and Hub (#5298)
• Standardize naming conventions (#5285)
• Fix metrics naming (#5277)
• Misc clean up (#5255, #5287, #5284, #5290, #5289, #5288)

🤟 Contributors

We would like to thank all contributors to this release:

• AlaeddineAbdessalem (@alaeddine-13)
• Anthony Le (@AnthonyLe93)
• felix-wang (@numb3r3)
• Girish Chandrashekar (@girishc13)
• zhangkai (@floralatin)
• Joan Fontanals (@JoanFM)
• Johannes Messner (@JohannesMessner)
• samsja (@samsja)
• Han Xiao (@hanxiao)
• Alex Cureton-Griffiths (@alexcg1)
• Andrei Ungureanu (@Andrei997)

Release Note (3.10.1)

Release time: 2022-10-06 15:05:48

This release contains 1 new feature, 1 refactor, 3 bug fixes and 7 documentation improvements.

🆕 Features

Distributed replicas across different hosts (#5217)

You can now start different Executors on different machines and use them as replicas by passing their addresses in the host parameter as a list, together with the external flag set to True:

f.add(host='localhost:12345,91.198.174.192:12346', external=True)

The Flow ensures the requests are load balanced between Executor instances.

⚙ Refactoring

Move Hubble related code to hubble-sdk (#5227)

All code related to hubble and communication with Jina Hub has been moved to the jina-hubble-sdk repo and has been added as an external dependency.

🐞 Bug Fixes

Disable rich traceback in logging (#5242)

We disabled the use of rich logging when printing tracebacks for Exceptions. This makes it easier to debug issues as the lines are not broken.

Previously, an Exception would be reported to the console as:

  ERROR  executor0/rep-0@183551 ZeroDivisionError('division by [10/05/22 11:26:23]
       zero')                                                                   
        add "--quiet-error" to suppress the exception                           
       details                                                                  
       ╭──────── Traceback (most recent call last) ────────╮                    
       │ /home/xxx/Documents/workspace/Jina/jina/jina/se… │                    
       │ in process_data                                   │                    
       │                                                   │                    
       │   179 │   │   │   │   if self.logger.debug_enable │                    
       │   180 │   │   │   │   │   self._log_data_request( │                    
       │   181 │   │   │   │                               │                    
       │ ❱ 182 │   │   │   │   result = await self._data_r │                    
       │   183 │   │   │   │   if self._successful_request │                    
       │   184 │   │   │   │   │   self._successful_reques │                    
       │   185 │   │   │   │   return result               │                    
       │                                                   │                    
       │ /home/xxx/Documents/workspace/Jina/jina/jina/se… │                    
       │ in handle                                         │                    
       │                                                   │                    
       │   161 │   │   )                                   │                    
       │   162 │   │                                       │                    
       │   163 │   │   # executor logic                    │                    
       │ ❱ 164 │   │   return_data = await self._executor. │                    
       │   165 │   │   │   req_endpoint=requests[0].header │                    
       │   166 │   │   │   docs=docs,                      │                    
       │   167 │   │   │   parameters=params,              │                    
       │                                                   │                    
       │ /home/xxx/Documents/workspace/Jina/jina/jina/se… │                    
       │ in __acall__                                      │                    
       │                                                   │                    
       │   290 │   │   if req_endpoint in self.requests:   │                    
       │   291 │   │   │   return await self.__acall_endpo │                    
       │   292 │   │   elif __default_endpoint__ in self.r │                    
       │ ❱ 293 │   │   │   return await self.__acall_endpo │                    
       │   294 │                                           │                    
       │   295 │   async def __acall_endpoint__(self, req_ │                    
       │   296 │   │   func = self.requests[req_endpoint]  │                    
       │                                                   │                    
       │ /home/xxx/Documents/workspace/Jina/jina/jina/se… │                    
       │ in __acall_endpoint__                             │                    
       │                                                   │                    
       │   311 │   │   │   if iscoroutinefunction(func):   │                    
       │   312 │   │   │   │   return await func(self, **k │                    
       │   313 │   │   │   else:                           │                    
       │ ❱ 314 │   │   │   │   return func(self, **kwargs) │                    
       │   315 │                                           │                    
       │   316 │   @property                               │                    
       │   317 │   def workspace(self) -> Optional[str]:   │                    
       │                                                   │                    
       │ /home/xxx/Documents/workspace/Jina/jina/jina/se… │                    
       │ in arg_wrapper                                    │                    
       │                                                   │                    
       │   159 │   │   │   │   def arg_wrapper(            │                    
       │   160 │   │   │   │   │   executor_instance, *arg │                    
       │   161 │   │   │   │   ):  # we need to get the su │                    
       │       the self                                    │                    
       │ ❱ 162 │   │   │   │   │   return fn(executor_inst │                    
       │   163 │   │   │   │                               │                    
       │   164 │   │   │   │   self.fn = arg_wrapper       │                    
       │   165                                             │                    
       │                                                   │                    
       │ /home/USER/.config/JetBrains/PyCharmCE2022.2/scr… │                    
       │ in foo                                            │                    
       │                                                   │                    
       │   12 │                                            │                    
       │   13 │   @requests                                │                    
       │   14 │   def foo(self, docs, **kwargs):           │                    
       │ ❱ 15 │   │   1/0                                  │                    
       │   16                                              │                    
       │   17 with Flow().add(uses=MyExec) as f:           │                    
       │   18                                              │                    
       ╰───────────────────────────────────────────────────╯                    
       ZeroDivisionError: division by zero                                      
Traceback (most recent call last):
  File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
    f.search(inputs=[Document()])
  File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
    return run_async(
  File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
    return asyncio.run(func(*args, **kwargs))
  File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
    async for resp in c._get_results(*args, **kwargs):
  File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
    callback_exec(
  File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
    raise BadServer(response.header)
jina.excepts.BadServer: request_id: "c50a57014fb948ccbd8065ada487a136"
status {
  code: ERROR
  description: "ZeroDivisionError(\'division by zero\')"
  exception {
    name: "ZeroDivisionError"
    args: "division by zero"
    stacks: "Traceback (most recent call last):\n"
    stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
    stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
    stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
    stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
    stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
    stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
    stacks: "ZeroDivisionError: division by zero\n"
    executor: "MyExec"
  }
}
exec_endpoint: "/search"
target_executor: ""


Process finished with exit code 1

After the fix, the error is much clearer:

ERROR  executor0/rep-0@198650 ZeroDivisionError('division by [10/05/22 11:40:37]
     zero')                                                                   
      add "--quiet-error" to suppress the exception                           
     details                                                                  
     Traceback (most recent call last):                                       
       File                                                                   
     "/home/USER/Documents/workspace/Jina/jina/jina/serve…                    
     line 182, in process_data                                                
         result = await                                                       
     self._data_request_handler.handle(requests=requests)                     
       File                                                                   
     "/home/USER/Documents/workspace/Jina/jina/jina/serve…                    
     line 164, in handle                                                      
         return_data = await self._executor.__acall__(                        
       File                                                                   
     "/home/USER/Documents/workspace/Jina/jina/jina/serve…                    
     line 293, in __acall__                                                   
         return await                                                         
     self.__acall_endpoint__(__default_endpoint__,                            
     **kwargs)                                                                
       File                                                                   
     "/home/USER/Documents/workspace/Jina/jina/jina/serve…                    
     line 314, in __acall_endpoint__                                          
         return func(self, **kwargs)                                          
       File                                                                   
     "/home/USER/Documents/workspace/Jina/jina/jina/serve…                    
     line 162, in arg_wrapper                                                 
         return fn(executor_instance, *args, **kwargs)                        
       File                                                                   
     "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratc…                    
     line 15, in foo                                                          
         1/0                                                                  
     ZeroDivisionError: division by zero                                      
Traceback (most recent call last):
File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
  f.search(inputs=[Document()])
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
  return run_async(
File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
  return asyncio.run(func(*args, **kwargs))
File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
  return loop.run_until_complete(main)
File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
  return future.result()
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
  async for resp in c._get_results(*args, **kwargs):
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
  callback_exec(
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
  raise BadServer(response.header)
jina.excepts.BadServer: request_id: "5d5e0338d0cd4bdd89efc430bbaaa74d"
status {
code: ERROR
description: "ZeroDivisionError(\'division by zero\')"
exception {
  name: "ZeroDivisionError"
  args: "division by zero"
  stacks: "Traceback (most recent call last):\n"
  stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
  stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
  stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
  stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
  stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
  stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
  stacks: "ZeroDivisionError: division by zero\n"
  executor: "MyExec"
}
}
exec_endpoint: "/search"
target_executor: ""


Process finished with exit code 1

Remove lambda function in logger (#5249)

We removed a reference to a lambda function in the JinaLogger so that there is no problem pickling it:

import pickle

from jina import Executor

class MyExecutor(Executor):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)


executor = MyExecutor()

with open('executor.pickle', 'wb') as f:
    pickle.dump(executor, f)

Previously, an Executor could not be pickled because it had a reference to a JinaLogger.

This used to raise an AttributeError exception:

Traceback (most recent call last):
  File "/home/USER/toy.py", line 14, in <module>
    pickle.dump(executor, f)
AttributeError: Can't pickle local object 'JinaLogger.__init__.<locals>.<lambda>'

Fix gRPC fork support (#5250)

We enabled proper fork support for gRPC to correctly clean up gRPC usage in parent and child processes. This prevents potential issues when forking Executor processes.

📗 Documentation Improvements

• Explain Executor patching in Kubernetes (#5235)
• Add note about keeping same Jina versions across microservices (#5239)
• Add missing request numbers in sample code (#5237)
• JCloud YAML reorder (#5234)
• Docs spelling/grammar/punctuation (#5244) (#5240)
• Fix typos (#5236)

🤟 Contributors

We would like to thank all contributors to this release:

• Sami Jaghouar (@samsja)
• Alex Cureton-Griffiths (@alexcg1)
• Johannes Messner (@JohannesMessner)
• Delgermurun (@delgermurun)
• AlaeddineAbdessalem (@alaeddine-13)
• Jackmin801 (@Jackmin801)
• Joan Fontanals (@JoanFM)
• Girish Chandrashekar (@girishc13)

Release Note (v3.10.0)

Release time: 2022-09-29 16:18:19

This release contains 10 new features, 9 bug fixes and 12 documentation improvements.

🆕 Features

Enable jina ping as readiness probe on local and Kubernetes (#5200)

This feature lets you check the readiness of an entire Flow or its individual components:

jina ping flow grpc://localhost:12345

jina ping executor localhost:12346

jina ping gateway grpc://localhost:12345

Successful output:

INFO   JINA@28600 readiness check succeeded 1 times!!! 

Unsuccessful output:

This command exits with code 1 when the readiness check is not successful. Therefore, it is a good choice to use as a readiness probe for Executor and Gateway when deployed in Kubernetes.

Add argument to specify exceptions that will exit the runtime (#5165)

You can specify which Exceptions should trigger termination of an affected Executor:

Flow().add(uses=MyExecutor, exit_on_exceptions: ['Exception',  'RuntimeException'])

This ensures an Executor is not stuck in an unusable but technically alive state, and that the Executor can be restarted. This is especially useful when Executor restarts are automated through Kubernetes.

Retry mechanism for client.post (#5176)

client.post() accepts max_attempts, initial_backoff, max_backoff and backoff_multiplier parameters to control the capacity to retry requests, when a transient connectivity error occurs, using an exponential backoff strategy:

client.post(
    on='/',
    inputs=docs,
    max_attempts=5,
    initial_backoff=0.1,
    max_backoff=0.5,
    backoff_multiplier=1.5
)

Add monitoring of requests size everywhere (#5111)

The size of requests (jina_request_size_bytes) is now monitored everywhere: On every Executor, every Gateway, and every sharding Head.

Consequently, this new metric can now be accessed via Prometheus and Grafana.

JSON logging for JCloud (#5201)

It is now possible to create logs in JSON format, instead of the standard line-by-line text output.

Sample output:

{"created": 1663930387.5850368, "filename": "data_request_handler.py", "funcName": "_load_executor", "levelname": "DEBUG", "lineno": 98, "module": "data_request_handler", "msg": "<jina.serve.executors.BaseExecutor object at 0x7fc1951f78d0> is successfully loaded!", "name": "WorkerRuntime", "pathname": "/home/foo/jina/serve/runtimes/request_handlers/data_request_handler.py", "process": 13386, "processName": "Pod", "thread": 140469860276032, "threadName": "MainThread"}

To enable this, set the environment variable 'JINA_LOG_CONFIG to 'json'.

Support list-like syntax to specify visible cuda devices (#5187)

You can now use device id's to specify visible CUDA devices that should be used in the round-robin GPU assignment:

`CUDA_VISIBLE_DEVICES=RR1,3`

This will assign CUDA devices 1 and 3 to the different Executor replicas in a round-robin fashion.

Make jina hub push non-blocking (#5129)

jina hub push is now a non-blocking operation, meaning that the user does not have to wait for the entire push to be completed on the server side.

While the push is being processed in the background, its status can be checked with the following command:

jina hub status [<path_to_executor_folder>] [--id TASK_ID] [--verbose] [--replay]

Warning messages in jina hub push (#5156)

jina hub push can now display helpful warning messages, such as deprecation warnings:

Add duration info in events (#5157)

Telemetry data now also includes the duration of every event that is reported.

🐞 Bug Fixes

Fix compatibility with Protobuf Python backend (#5222)

This bug caused an exception when Python was used as a backend for Protobuf, rather than cpp or udp.

With this fix, Python can also be used as a backend for Protobuf.

Move significant params to sequential (#5205)

This bug (occasionally) caused docker.errors.NullResource: Resource ID was not provided when pulling from Jina Hub. This was due to a faulty local caching implementation, which is now fixed.

Fix missing secret when logged-in user with --force-update (#5180)

This bug caused a failure for logged in users when force pushing to Jina Hub: Their secret would not be passed. This is now fixed and force push works as expected.

Use default gRPC parameters for gRPC connection pool connection (#5211)

We brought our gRPC client in line with the recommended configuration at https://github.com/grpc/grpc/blob/master/doc/keepalive.md

Bump DocArray version for new column syntax (#5171)

Some changes to the column syntax in DocArray required a version bump.

Increase minimum Protobuf version (#5166)

Some changes in the Protobuf library required a version bump.

Remove leftover prints (#5199)

Some unwanted print statements had sneaked their way into our codebase, and we kicked them right back out!

📗 Documentation Improvements

• Change setenv into environ (#5223)
• Mention prefetch (#5220)
• Fix install instructions docker (#5213)
• Fix grammar, wording, punctuation (#5209)
• Fix grammar, punctuation (#5208)
• Clarify exec endpoint usage in http (#5202)
• Document grpc client limitation (#5193)
• Update faq and lifetime (#5191)
• Improve warning about config file in custom docker (#5190)
• Clarify return behavior in parameters (#5179)
• Add section for exit_on_exceptions argument (#5172)
• Labels in flow yaml (#5164)

🤟 Contributors

We would like to thank all contributors to this release:

• AlaeddineAbdessalem (@alaeddine-13)
• Girish Chandrashekar (@girishc13)
• Andrei Ungureanu (@Andrei997)
• Michael Günther (@guenthermi)
• Deepankar Mahapatro (@deepankarm)
• Han Xiao (@hanxiao)
• zhangkai (@floralatin)
• Joan Fontanals (@JoanFM)
• Alex Cureton-Griffiths (@alexcg1)
• Yanlong Wang (@nomagick)
• Nikolas Pitsillos (@npitsillos)
• samsja (@samsja)

Release Note (3.9.3)

Release time: 2022-09-29 16:05:04

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, AlaeddineAbdessalem, Michael Günther, Yanlong Wang, samsja, Alex Cureton-Griffiths, Deepankar Mahapatro, Nikolas Pitsillos, Han Xiao, zhangkai, Jina Dev Bot, 🙇

🆕 New Features

• [11533962] - add gateway option to jina ping and use ping in readinessProbe (#5200) (Joan Fontanals)
• [d0838d37] - add multiple attempts options to client.post API (#5176) (Joan Fontanals)
• [13edf908] - use default grpc parameters for grpc connection pool connection (#5211) (samsja)
• [216b4bf0] - monitoring: add monitoring of requests size in bytes at all level (#5111) (samsja)
• [737875f5] - logs: json logging for jcloud (#5201) (Deepankar Mahapatro)
• [fd4c0347] - support list-like syntax to round robin CUDA devices (#5187) (Joan Fontanals)
• [9dff4c88] - add duration info in events (#5157) (Joan Fontanals)

🐞 Bug fixes

• [3683fccf] - fix compatibility with protobuf python backend (#5222) (AlaeddineAbdessalem)
• [e932d6f9] - hubio.fetch_meta: move significant params to sequential (#5205) (Yanlong Wang)
• [60e9b8de] - remove leftover prints (#5199) (Joan Fontanals)
• [da186a23] - provide logger and streamer (#5188) (AlaeddineAbdessalem)
• [7b5c0316] - fix get-openapi-schemas script (#5185) (AlaeddineAbdessalem)
• [caf4a3d6] - fix missing secret when logged-in user with --force-update and … (#5180) (zhangkai)

🧼 Code Refactoring

• [9a6079ef] - separate gateway and asyncio runtime readiness checks (#5224) (AlaeddineAbdessalem)
• [243639dd] - extract gateway app logic into custom gateway class (#5153) (AlaeddineAbdessalem)

📗 Documentation

• [147317db] - change setenv into environ (#5223) (Michael Günther)
• [ad37bdd2] - mention prefetch (#5220) (Joan Fontanals)
• [b9c8e44e] - fix install instructions docker (#5213) (Joan Fontanals)
• [7cbf3471] - what-is-jina: fix grammar, wording, punctuation (#5209) (Alex Cureton-Griffiths)
• [7c7d2cb0] - what-is-modality: fix grammar, punctuation (#5208) (Alex Cureton-Griffiths)
• [1399e36c] - clarify exec endpoint usage in http (#5202) (Joan Fontanals)
• [842a585b] - document grpc client limitation (#5193) (AlaeddineAbdessalem)
• [96c5e7c1] - jcloud: update faq and lifetime (#5191) (Nikolas Pitsillos)
• [c17e6416] - improve warning about config file in custom docker (#5190) (Joan Fontanals)
• [22330fcb] - clarify return behavior in parameters (#5179) (Joan Fontanals)

🍹 Other Improvements

• [b5977f47] - update contributing to remove labeling (#5225) (Joan Fontanals)
• [45c7c6e1] - update pull request template link (#5214) (Joan Fontanals)
• [1eab9ce6] - fix doc template (Han Xiao)
• [8c2d0758] - docs: update TOC (Jina Dev Bot)
• [139b068e] - version: the next version will be 3.9.3 (Jina Dev Bot)

Release Note (3.9.2)

Release time: 2022-09-15 14:00:36

🙇 We'd like to thank all contributors for this new release! In particular, Girish Chandrashekar, zhangkai, AlaeddineAbdessalem, Joan Fontanals, Jina Dev Bot, Andrei Ungureanu, Deepankar Mahapatro, 🙇

🆕 New Features

• [c51f9014] - hubble async push (#5129) (zhangkai)
• [3f39ed46] - runtime: add argument to specify exceptions that will exit the runtime (#5165) (Girish Chandrashekar)
• [27e1f779] - hubio: display warning messages from hubble request (jina hub push) (#5156) (Andrei Ungureanu)

🐞 Bug fixes

• [29ad1750] - exit on exception args only applies to executors (#5169) (AlaeddineAbdessalem)
• [55188165] - pin docarray version for new column syntax (#5171) (Joan Fontanals)
• [fa83955c] - increase minimum protobuf version (#5166) (AlaeddineAbdessalem)

📗 Documentation

• [28aeac8e] - add section for exit_on_exceptions argument (#5172) (Girish Chandrashekar)
• [33891c46] - jcloud: labels in flow yaml (#5164) (Deepankar Mahapatro)

🏁 Unit Test and CICD

• [b59d0450] - add jina auth token (#5167) (AlaeddineAbdessalem)

🍹 Other Improvements

• [76dca020] - update announcement in readme and docs (Jina Dev Bot)
• [f4248ea8] - docs: update TOC (Jina Dev Bot)
• [fda5623a] - version: the next version will be 3.9.2 (Jina Dev Bot)

Release Note (3.9.1)

Release time: 2022-09-08 14:10:44

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🧼 Code Refactoring

• [273fda5a] - merge dryrun into ping (#5151) (Han Xiao)

🍹 Other Improvements

• [0a71009b] - version: the next version will be 3.9.1 (Jina Dev Bot)

Highlights :star2:

Update to new protobuf version

We have updated our protobuf dependency to use the latest protobuf version leveraging all the performance improvement that comes with it.

#5082

Merge manifest.yml and config.yml for Jina Hub Executors

We have made it easier for the users to manage their Hub Executors by merging the information from manifest.yml to config.yml which makes the experience much better.

#5101

Expose grpc server parameters to Executor

We expose all the parameters that grpc server can accept for users to be able to configure their services in the most robust and reliable way.

#5092

Other changes

• Add more Dockerfile templates options in jina hub new #5104
• Add failed and successfull requests metrics #5079

Bug fixes

• Fix reading empty list from environment variables #5143

Release Note (3.8.4)

Release time: 2022-09-08 08:51:32

🙇 We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, AlaeddineAbdessalem, Joan Fontanals, Johannes Messner, Andrei Ungureanu, samsja, Jina Dev Bot, 🙇

🆕 New Features

• [d1faf6e6] - pass internal flag to telemetry (#5134) (Joan Fontanals)

🐞 Bug fixes

• [e501ddfd] - version: adapt to docker hub v2 (#5146) (Deepankar Mahapatro)
• [d76e69ac] - fix parsing empty list (#5143) (AlaeddineAbdessalem)
• [3ec33c08] - gpu dockerfile template include now jina install (#5124) (samsja)
• [e2967f98] - set minimal docarray dependency (#5133) (Joan Fontanals)
• [ff803686] - linkerd cd (#5131) (AlaeddineAbdessalem)

🧼 Code Refactoring

• [a78612f3] - manifest config (#5101) (Andrei Ungureanu)

🏁 Unit Test and CICD

• [ddf90efb] - remove outdated test (#5139) (Johannes Messner)
• [a4ea7215] - increase wait time (#5137) (AlaeddineAbdessalem)
• [478e53f6] - fix k8s tests return responses params (#5127) (Joan Fontanals)

🍹 Other Improvements

• [a9b80aec] - docs: update TOC (Jina Dev Bot)
• [1df9b459] - version: the next version will be 3.8.4 (Jina Dev Bot)

Release Note (3.8.3)

Release time: 2022-09-05 14:48:30

🙇 We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Jina Dev Bot, 🙇

🐞 Bug fixes

• [7b7eecc5] - cap grpc version (#5122) (AlaeddineAbdessalem)

🍹 Other Improvements

• [47e2cfc3] - docs: update TOC (Jina Dev Bot)
• [0dbdcaa0] - version: the next version will be 3.8.3 (Jina Dev Bot)

Release Note (3.8.2)

Release time: 2022-09-03 15:56:11

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Zac Li, samsja, AlaeddineAbdessalem, Jina Dev Bot, tarrantro, 🙇

🆕 New Features

• [e794c06c] - expose grpc parameters and add production ready keepalive parameters (#5092) (samsja)
• [6205ffc9] - add gpu dockerfile support to jina hub new (#5104) (AlaeddineAbdessalem)

📗 Documentation

• [9293698b] - rewrite flow add (Han Xiao)
• [115bc0fa] - rewrite flow add (#5120) (Han Xiao)
• [22fdbcee] - jcloud: fix jc status link (#5116) (Zac Li)
• [4dfe6dfb] - jcloud: fix docs for gateway resource customization (#5114) (Zac Li)
• [6b24bf69] - correct jcloud retention days (#5109) (tarrantro)

🏁 Unit Test and CICD

• [587f6268] - fix test concurrent async flow (#5115) (AlaeddineAbdessalem)

🍹 Other Improvements

• [763f6773] - update readme (Han Xiao)
• [2651c506] - disable hub-integration (Han Xiao)
• [3590c385] - fix readme (Han Xiao)
• [8303fb25] - docs: improve docs on jcloud k8s (Han Xiao)
• [a34edc62] - update announcement in readme and docs (Jina Dev Bot)
• [d124d9af] - docs: update TOC (Jina Dev Bot)
• [0df7cd8a] - version: the next version will be 3.8.2 (Jina Dev Bot)

Release Note (3.8.1)

Release time: 2022-08-30 19:32:52

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, AlaeddineAbdessalem, Jina Dev Bot, 🙇

🆕 New Features

• [6df36ce5] - client: add profiling function to client and flow (#5105) (Han Xiao)

🐞 Bug fixes

• [78aae025] - fix inconsistent docs and tests on py_modules (#5108) (Han Xiao)
• [a725010d] - remove flakyness of monitoring integration tests in CI (#5106) (samsja)

🧼 Code Refactoring

• [90fa81f3] - move port monitoring to pods parsing (#5100) (samsja)

📗 Documentation

• [95ee283c] - fix naming (#5102) (AlaeddineAbdessalem)

🍹 Other Improvements

• [d528d947] - fix cd pipeline (Han Xiao)
• [57625350] - docs: update TOC (Jina Dev Bot)
• [1ad06216] - docs: fix retention days description (Han Xiao)
• [9f5f5879] - version: the next version will be 3.8.1 (Jina Dev Bot)

Release Note (3.8.0)

Release time: 2022-08-29 16:43:11

🙇 We'd like to thank all contributors for this new release! In particular, Johannes Messner, tarrantro, Han Xiao, AlaeddineAbdessalem, samsja, Jina Dev Bot, Zac Li, 🙇

🆕 New Features

• [527beb85] - bump protobuf version (#5082) (AlaeddineAbdessalem)
• [c47cb716] - add failed and successful request number metrics (#5079) (AlaeddineAbdessalem)

🐞 Bug fixes

• [c81252ac] - update to protobuf 4.21 new types (#5098) (Johannes Messner)

🧼 Code Refactoring

• [6e51e815] - refactor data request handler monitoring (#5088) (samsja)

📗 Documentation

• [b3fdb49c] - correct jcloud retention days (#5096) (tarrantro)
• [98fb71f1] - jcloud: adjust jcloud monitoring docs(#5081) (Zac Li)
• [dbea43f3] - remove kong/alb from jcloud document (#5080) (tarrantro)

🏁 Unit Test and CICD

• [27ec3b41] - cleanup pip install (#5094) (AlaeddineAbdessalem)
• [c0c3bf9a] - downgrade linkerd version in CI (#5090) (AlaeddineAbdessalem)

🍹 Other Improvements

• [142c0bb1] - docs: add streamline value props (Han Xiao)
• [af7f6763] - fix readme (Han Xiao)
• [c39714fa] - docs: add install apple silicon (Han Xiao)
• [5375b4d3] - update announcement in readme and docs (Jina Dev Bot)
• [53553b14] - add security report (Han Xiao)
• [cfe5dd70] - update readme (Han Xiao)
• [e106796c] - docs: update furo deps (Han Xiao)
• [079d04b0] - docs: add jcloud to first app (Han Xiao)
• [64de8608] - docs: add comparing alternatives (Han Xiao)
• [f9c2d8de] - docs: update TOC (Jina Dev Bot)
• [2154f98f] - version: the next version will be 3.7.15 (Jina Dev Bot)

Release Note (3.7.14)

Release time: 2022-08-22 13:52:20

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🍹 Other Improvements

• [4f380c7e] - fix pprint (Han Xiao)
• [0bd289fe] - docs: update TOC (Jina Dev Bot)
• [96489914] - version: the next version will be 3.7.14 (Jina Dev Bot)

Release Note (3.7.13)

Release time: 2022-08-21 21:03:25

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Johannes Messner, Jina Dev Bot, 🙇

🐞 Bug fixes

• [b7bf0ee1] - set workspace in executor init method (#5072) (Johannes Messner)

🧼 Code Refactoring

• [f6dcde38] - telemetry (#5078) (Han Xiao)

🍹 Other Improvements

• [8ad0ead5] - docs: refactor create project docs (Han Xiao)
• [c7e19841] - update readme (Han Xiao)
• [7b034d34] - docs: refactor telemetry docs (Han Xiao)
• [76f48f8f] - docs: refactor jcloud docs (Han Xiao)
• [e506da44] - docs: refactor hub docs (Han Xiao)
• [b91f2084] - docs: refactor executors docs (Han Xiao)
• [f8a63ccf] - docs: refactor flow basic docs (Han Xiao)
• [885914ad] - docs: restructure docs (Han Xiao)
• [36fdebe2] - docs: fix docs in jcloud (Han Xiao)
• [4673e7ab] - docs: update TOC (Jina Dev Bot)
• [a4c48d1b] - version: the next version will be 3.7.13 (Jina Dev Bot)

Release Note (3.7.12)

Release time: 2022-08-18 22:23:25

🙇 We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Johannes Messner, Jina Dev Bot, 🙇

🆕 New Features

• [6f87524c] - include jcloud cli (#5074) (AlaeddineAbdessalem)

🍹 Other Improvements

• [10c63db2] - update pr template with reminder to add a description (#5062) (Johannes Messner)
• [6fbd2be9] - docs: update TOC (Jina Dev Bot)
• [7d82fdc3] - version: the next version will be 3.7.12 (Jina Dev Bot)

Release Note (3.7.11)

Release time: 2022-08-17 21:09:48

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, 🙇

🐞 Bug fixes

• [15fd60d5] - cli: jina auth cli error (Han Xiao)

🍹 Other Improvements

• [80393316] - version: the next version will be 3.7.11 (Jina Dev Bot)

Release Note (3.7.10)

Release time: 2022-08-17 12:22:49

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Jina Dev Bot, 🙇

🐞 Bug fixes

• [b9bddd22] - cli: jina auth cli error (#5070) (Han Xiao)

📗 Documentation

• [840c221f] - add gateway timeout and update shared gpu (#5069) (tarrantro)

🍹 Other Improvements

• [bffe263c] - docs: update TOC (Jina Dev Bot)
• [94c83cdb] - version: the next version will be 3.7.10 (Jina Dev Bot)

Release Note (3.7.9)

Release time: 2022-08-16 16:25:11

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, AlaeddineAbdessalem, Jina Dev Bot, 🙇

🐞 Bug fixes

• [26d24d03] - #5055 (#5066) (Han Xiao)

🧼 Code Refactoring

• [c1c500aa] - remove unused hub environment variables (#5067) (AlaeddineAbdessalem)

🍹 Other Improvements

• [442a86cd] - version: the next version will be 3.7.9 (Jina Dev Bot)

Release Note (3.7.8)

Release time: 2022-08-16 15:18:50

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, Zac Li, 🙇

🧼 Code Refactoring

• [bf864748] - hub: use hubble sdk (#5064) (Han Xiao)

📗 Documentation

• [317e9ce1] - jcloud: doc for gateway customization (#5065) (Zac Li)

🍹 Other Improvements

• [a6be9481] - update announcement in readme and docs (Jina Dev Bot)
• [f3bcdaa8] - docs: update TOC (Jina Dev Bot)
• [dfd1e07d] - docs: fix typos (Han Xiao)
• [d579cb63] - version: the next version will be 3.7.8 (Jina Dev Bot)

Release Note (3.7.7)

Release time: 2022-08-15 18:14:16

🙇 We'd like to thank all contributors for this new release! In particular, Johannes Messner, Han Xiao, Jina Dev Bot, 🙇

🐞 Bug fixes

• [22563074] - cache hub pull with build env (#5061) (Johannes Messner)

🍹 Other Improvements

• [e7377947] - docs: fix typos (Han Xiao)
• [6d218fbf] - version: the next version will be 3.7.7 (Jina Dev Bot)

Release Note (3.7.6)

Release time: 2022-08-15 14:04:25

🙇 We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Zac Li, Jina Dev Bot, 🙇

🧼 Code Refactoring

• [49a842d9] - unify temp path to dot cache jina (#5058) (Han Xiao)

📗 Documentation

• [731d492c] - add CPU and GPU type (#5060) (tarrantro)
• [d8f6f418] - rewrite the telemetry session (#5059) (Han Xiao)
• [01111539] - jcloud: autoscale docs (#5056) (Zac Li)

🍹 Other Improvements

• [299a908b] - docs: add what is jina (Han Xiao)
• [23914a81] - add what is cross multi modal (Han Xiao)
• [0c78b993] - docs: update TOC (Jina Dev Bot)
• [102bc6de] - version: the next version will be 3.7.6 (Jina Dev Bot)

Release Note (3.7.5)

Release time: 2022-08-10 10:19:23

🙇 We'd like to thank all contributors for this new release! In particular, Jina Dev Bot, zhangkai, samsja, Deepankar Mahapatro, tarrantro, 🙇

🆕 New Features

• [6cc4497a] - hubble support env variables (#5023) (zhangkai)
• [12404535] - add dryrun to cli (#5050) (samsja)

📗 Documentation

• [1d60655d] - jcloud: fix formatting (#5048) (Deepankar Mahapatro)
• [6df83cf1] - improve grpc third party client documentation (#5047) (samsja)
• [8c6321ac] - refactor resources, add gpu, kong (#5027) (tarrantro)

🍹 Other Improvements

• [d6d4c30a] - update announcement in readme and docs (Jina Dev Bot)
• [9fee2dff] - docs: update TOC (Jina Dev Bot)
• [fe7b8894] - version: the next version will be 3.7.5 (Jina Dev Bot)

Release Note (3.7.4)

Release time: 2022-08-03 12:31:57

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

• [4c3d760f] - do not send target_executor to Executors (#5041) (Joan Fontanals)

🍹 Other Improvements

• [a51b42cc] - update announcement in readme and docs (Jina Dev Bot)
• [ecb95829] - docs: update TOC (Jina Dev Bot)
• [286bdce8] - version: the next version will be 3.7.4 (Jina Dev Bot)

Release Note (3.7.3)

Release time: 2022-07-28 14:29:23

🙇 We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, 🙇

🐞 Bug fixes

• [032bd5e6] - fix specific params problem with branches (#5038) (Joan Fontanals)

🍹 Other Improvements

• [81999e97] - version: the next version will be 3.7.3 (Jina Dev Bot)

Release Note (3.7.2)

Release time: 2022-07-28 09:19:06

🙇 We'd like to thank all contributors for this new release! In particular, Zac Li, Jina Dev Bot, 🙇

🐞 Bug fixes

• [fb88015e] - add more logging in grpc dry run (#5036) (Zac Li)

🍹 Other Improvements

• [5116b922] - docs: update TOC (Jina Dev Bot)
• [32dd47af] - version: the next version will be 3.7.2 (Jina Dev Bot)