Finetuner Update 0.7.2
Finetuner makes neural network fine-tuning easier and faster by streamlining the workflow and handling all the complexity and infrastructure requirements in the cloud.
Finetuner makes neural network fine-tuning easier and faster by streamlining the workflow and handling all the complexity and infrastructure requirements in the cloud. With Finetuner, one can easily enhance the performance of pre-trained models and make them production-ready without expensive hardware.
jina-aiThis release covers Finetuner version 0.7.2, including dependencies finetuner-api 0.5.2 and finetuner-core 0.12.7.
This release contains 2 new features, 4 refactorings, 1 bug fix, and 3 documentation improvements.
🆕 Features
Support learning rate scheduler (#679)
This PR add supports for the learning rate scheduler. The scheduler is used to adjust the learning rate during training. We support 6 learning rate schedulers: linear, cosine, cosine_with_restarts, polynomial, constant and constant_with_warmup.
When a scheduler is configured, the learning rate is by default adjusted after each batch. Alternatively, you can set scheduler_optons = {'scheduler_step': 'epoch'} to adjust the learning rate after each epoch instead.
You can use them by specifying their name in the scheduler attribute of the fit function.
run = finetuner.fit(
...,
scheduler='linear',
scheduler_options={'scheduler_step': 'batch'},
...
)
Support steps_per_interval in EvaluationCallback
When working with large datasets, you may want to perform evaluations multiple times during each epoch. This parameter allows to specify a number of batches after which an evaluation should be performed. If set to None, an evaluation is performed only at the end of each epoch.
run = finetuner.fit(
...,
callbacks=[
EvaluationCallback(
query_data=...,
index_data=...,
steps_per_interval=3, # evaluate every 3 batches.
),
],
...
)
⚙ Refactoring
scheduler_step becomes part of scheduler_options (#679)
We removed the scheduler_step argument from the fit function, now it is part of the scheduler_options.
run = finetuner.fit(
...,
scheduler='linear',
- scheduler_step='batch',
+ scheduler_options={'scheduler_step': 'batch'},
...
)
Change the default epochs and batch_size in cloud.jina.ai
For Web UI users, we have reduced the default epochs from 10 to 5, and reduced the default batch_size from 128 to 64 to avoid out-of-memory errors from 3D-mesh fine-tuning.
Improve the user journey in cloud.jina.ai
Add more textual guidance on creating Finetuner runs in the Jina AI Cloud UI.
Remove duplicate query-document pairs in unlabeled CSVs (#678)
Finetuner now groups query-document pairs by their queries, thereby eliminating duplicate queries, when parsing CSV files. This leads to more effective fine-tuning.
🐞 Bug Fixes
Remove invalid argument from GeM pooler.
This PR removes the output_dim argument from the GeM pooler's forward function. You can use GeM pooler together with ArcFaceLoss to deliever better visual embedding quality.
run = finetuner.fit(
...,
model_options = {
...
- 'output_dim': 512,
+ 'pooler': 'GeM',
+ 'pooler_options': {'p': 2.4, 'eps': 1e-5}
}
)
📗 Documentation Improvements
Add a documentation section for GeM pooling (#684)
We have added a new section to our documentation which explains the pooling options in more detail.
Add a documentation page and notebook for ArcFaceLoss (#680)
We have added a new page to our documentation which demonstrates ArcFaceLoss on the Stanford Cars dataset.
Add a documentation section on creating query-document pairs as data (#678)
We have added a new section to our documentation explaining how to create training data made of query-document pairs instead of explicitly annotated and labeled data.
🤟 Contributors
We would like to thank all contributors to this release:
- Wang Bo (@bwanglzu)
- Louis Milliken (@LMMilliken)
- Michael Günther (@guenthermi)
- CatStark (@CatStark)
- George Mastrapas (@gmastrapas)
- Martin Matousek (@matousek-martin)
- Scott Martens (@scott-martens)