Releases

Jina 3.18 Update

Jina is a MLOps framework that empowers anyone to build cross-modal and multi-modal applications on the cloud.

Engineering Group

Jun 22, 2023 • 3 min read

Release Note (`3.18.0`)

This release contains 2 new features, 4 bug fixes, and 2 documentation improvements.

🆕 Features

Streaming single Document with HTTP SSE for Deployment (#5899)

In this release, we have added support for Server-Sent Events (SSE) to Jina's HTTP protocol with the Deployment orchestration. A Jina server can now stream single Documents to a client, one at a time. This is useful for applications that require a continuous stream of data, such as chatbots (using Large Language Models) or real-time translation.

Simply define an endpoint function that receives a single Document and yields Documents one by one:

from jina import Executor, requests, Document, Deployment

class MyExecutor(Executor):
    @requests(on='/hello')
    async def task(self, doc: Document, **kwargs):
        for i in range(3):
            yield Document(text=f'{doc.text} {i}')
            
with Deployment(
    uses=MyExecutor,
    port=12345,
    protocol='http',
    cors=True,
    include_gateway=False,
) as dep:
    dep.block()

From the client side, you can use the new stream_doc method to receive the Documents one by one:

from jina import Client, Document

client = Client(port=12345, protocol='http', cors=True, asyncio=True)
async for doc in client.stream_doc(
    on='/hello', inputs=Document(text='hello world')
):
    print(doc.text)

Note that the SSE client is language-independent. This feature also supports DocArray v2. For more information, see the streaming endpoints section of the Jina documentation.

Add `env_from_secret` option to `Gateway` (#5914)

We have added the env_from_secret parameter to Gateway to allow custom gateways to load secrets from Kubernetes when transformed to Kubernetes YAML in the same way as Executors.

from jina import Flow

f = Flow().config_gateway(env_from_secret={'SECRET_PASSWORD': {'name': 'mysecret', 'key': 'password'}}).add()
f.to_kubernetes_yaml()

🐞 Bug Fixes

Fix error working with some data types in DocArray V2 (#5905) (#5908)

We have fixed some errors when DocArray v2 documents contained flexible dictionaries, Lists, or Tensors.

Fix reloading Executor when is loaded from `config.yml` (#5915)

The reload option of an Executor was not working when the Executor was loaded from config.yml, and Jina was not able to update Executor code after updates.

f = Flow().add(uses='config.yml', reload=True)
with f:
    f.block()

Ensure closing `Executor` at shutdown in `Deployment` with HTTP protocol (#5906)

We have fixed a bug that prevented the close method of the Executor from being executed at shutdown when the Deployment is exposed with an HTTP server.

Fix issue in mismatch endpoint when using shards (#5904)

A KeyError was raised when working with DocArray v2 in a Deployment using shards if the endpoints were not matching. With this fix, it will properly call the default endpoint.

from jina import Deployment, Executor, requests
from docarray import BaseDoc, DocList
from docarray.typing import NdArray
fromt typing import List

class MyDoc(BaseDoc):
    text: str
    embedding: NdArray[128]

class MyDocWithMatchesAndScores(MyDoc):
    matches: DocList[MyDoc]
    scores: List[float]

class MyExec(Executor):

    @requests
    def foo(self, docs: DocList[MyDoc], **kwargs) -> DocList[MyDocWithMatchesAndScores]:
        res = DocList[MyDocWithMatchesAndScores]()
        for doc in docs:
            new_doc = MyDocWithMatchesAndScores(text=doc.text, embedding=doc.embedding, matches=docs,
                                                scores=[1.0 for _ in docs])
            res.append(new_doc)
        return res

d = Deployment(uses=MyExec, shards=2)
with d:
    res = d.post(on='/', inputs=DocList[MyDoc]([MyDoc(text='hey ha', embedding=np.random.rand(128))]))
    assert len(res) == 1

📗 Documentation Improvements

README revamp to put more emphasis on generative AI use cases (#5895)
Fix links in docs (#5909)

🤟 Contributors

We would like to thank all contributors to this release:

Alex Cureton-Griffiths (@alexcg1 )
AlaeddineAbdessalem (@alaeddine-13)
Joan Fontanals (@JoanFM )

Release Note (3.18.0)