Sign in Subscribe
Releases

Jina 3.15 Update

Jina is a MLOps framework that empowers anyone to build cross-modal and multi-modal applications on the cloud.

Engineering Group

• 5 min read
Promotional graphic for a microchip's new release 3.15, featuring an abstract design with a black background, orange and blue

Release Note (3.15.0)

Release time: 2023-04-14 09:55:25

This release contains 6 new features,  6 bug fixes, and 5 documentation improvements.

Release 💫 Release v3.15.0 · jina-ai/jina
Release Note (3.15.0) Release time: 2023-04-14 09:55:25 This release contains 6 new features, 6 bug fixes and 5 documentation improvements.🆕 FeaturesHTTP and composite protocols for Deployment…
GitHubjina-ai

🆕 Features

HTTP and composite protocols for Deployment (#5764)

When using a Deployment to serve a single Executor, you can now expose it via HTTP or a combination of HTTP and gRPC protocols:

from jina import Deployment, Executor, requests

class MyExec(Executor):

    @requests(on='/bar')
    def bar(self, docs, **kwargs):
        pass

dep = Deployment(protocol=['http', 'grpc'], port=[12345, 12346], uses=MyExec)

with dep:
    dep.block()

With this, you can also access the OpenAPI schema vialocalhost:12345/docs:

http-swagger-ui

Force network mode option (#5789)

When using a containerized Executor inside a Deployment or as part of a Flow, under some circumstances you may want to force the network mode to make sure the container is reachable by the Flow or Deployment to ensure readiness. This ensures that the Docker Python SDK runs the container with the relevant options.

For this, we have added the argument force_network_mode to enable this.

You can set this argument to any of these options:

  • AUTO: Automatically detect the Docker network.
  • HOST: Use the host network.
  • BRIDGE: Use a user-defined bridge network.
  • NONE: Use None as the network.
from jina import Deployment

dep = Deployment(uses='jinaai+docker://TransformerTorchEncoder', force_network_mode='None')

with dep:
    dep.block()

Allow disabling thread lock (#5771)

When an Executor exposes a synchronous method (not a coroutine) and exposes this method via the @requests decorator, Jina makes sure that each request received is run in a thread.

However, this thread is locked with a threading.Lock object to protect the user from potential hazards of multithreading while leaving the Executor free to respond to health checks coming from the outside or from orchestrator frameworks such as Kubernetes. This lock can now be bypassed if the allow_concurrent argument is passed to the Executor.

from jina import Deployment, Executor, requests

class MyExec(Executor):

    @requests(on='/bar')
    def bar(self, docs, **kwargs):
        pass

dep = Deployment(allow_concurrent=True, uses=MyExec)

with dep:
    dep.block()

grpc_channel_options for custom gRPC options for the channel (#5765)

You can now pass grpc_channel_options to allow granular tuning of the gRPC connectivity from the Client or Gateway. You can check the options in the gRPC Python documentation.

client = Client(grpc_channel_options={'grpc.max_send_message_length': -1})

Create Deployments from the CLI (#5756)

New you can create from the Jina CLI to create a first project to deploy a single Deployment in the same way it was possible to create one for a Flow.

Now the jina new command accepts a new argument type that can be flow or deployment.

jina new hello-world --type flow
jina new hello-world --type deployment

Add replicas argument to Gateway for Kubernetes (#5711)

To scale the Gateway in Kubernetes or in JCloud, you can now add the replicas arguments to the gateway section.

from jina import Flow

f = Flow().config_gateway(replicas=3).add()
f.to_kubernetes_yaml('./k8s_yaml_path')
jtype: Flow
version: '1'
with: {}
gateway:
  replicas: 3
executors:
- name: executor0

🐞 Bug Fixes

Retry client gRPC stream and unary RPC methods (#5733)

The retry mechanism parameters were not properly respected by the Client in prior releases. This is now fixed and will improve robustness against transient errors.

from jina import Client, DocumentArray

Client(host='...').post(
    on='/',
    inputs=DocumentArray.empty(),
    max_attempts=100,
)

Allow HTTP timeout (#5797)

When using the Client to send data to an HTTP service, the connection timed out after five minutes (the default setting for aiohttp). This can now be modified with the timeout argument for cases where a request may take longer, thus avoiding the Client disconnecting after a longer period.

from jina import Client, DocumentArray

Client(protocol='http').post(
    on='/',
    inputs=DocumentArray.empty(),
    timeout=600,
)

Enable root logging at all times (#5736)

The JINA_LOG_LEVEL environment variable controls the log level of the JinaLogger. Previously the debug logging of other dependencies was not respected. Now they can be enabled.

logging.get_logger('urllib3').setLevel(logging.DEBUG)

Fix Gateway tensor serialization (#5752)

In prior releases, when an HTTP Gateway was run without torch installed and connected to an Executor returning torch.Tensor as part of the Documents, the Gateway couldn't serialize the Documents back to the Client, leading to a no module torch error. This is now fixed and works without installing torch in the Gateway container or system.

from jina import Flow, Executor, Document, DocumentArray, requests

import torch

class DummyTorchExecutor(Executor):
    @requests
    def foo(self, docs: DocumentArray, **kwargs):
        for d in docs:
            d.embedding = torch.rand(1000)
            d.tensor = torch.rand(1000)
from jina import Flow, Executor, Document, DocumentArray, requests

f = Flow().config_gateway(port=12346, protocol='http').add(port='12345', external=True)

with f:
    docs = f.post(on='/', inputs=Document())
    print(docs[0].embedding.shape)
    print(docs[0].tensor.shape)

Composite Gateway tracing (#5741)

Previously, tracing didn't work for Gateways that exposed multiple protocols:

from jina import Flow

f = Flow(port=[12345, 12346], protocol=['http', 'grpc'], tracing=True).add()

with f:
   f.block()

Adapt to DocArray v2 (#5742)

Jina depends on DocArray's data structures. This version adds support for DocArray v2's upcoming major changes.

The involves naming conventions:

  • DocumentArray➡DocList
  • BaseDocument➡ BaseDoc
from jina import Deployment, Executor, requests
from docarray import DocList, BaseDoc
from docarray.documents import ImageDoc
from docarray.typing import AnyTensor

import numpy as np

class InputDoc(BaseDoc):
    img: ImageDoc

class OutputDoc(BaseDoc):
    embedding: AnyTensor

class MyExec(Executor):
    @requests(on='/bar')
    def bar(
        self, docs: DocList[InputDoc], **kwargs
    ) -> DocumentArray[OutputDoc]:
        docs_return = DocList[OutputDoc](
            [OutputDoc(embedding=np.zeros((100, 1))) for _ in range(len(docs))]
        )
        return docs_return

with Deployment(uses=MyExec) as dep:
    docs = dep.post(
        on='/bar',
        inputs=InputDoc(img=ImageDoc(tensor=np.zeros((3, 224, 224)))),
        return_type=DocList[OutputDoc],
    )
    assert docs[0].embedding.shape == (100, 1)
    assert docs.__class__.document_type == OutputDoc

📗 Documentation Improvements

  • JCloud Flow name customization (#5778)
  • JCloud docs revamp for instance (#5759)
  • Fix Colab link (#5760)
  • Remove docsQA (#5743)
  • Miscellaneous polishing

🤘 Contributors

We would like to thank all contributors to this release:

Sign up for more like this.

Enter your email
Subscribe