Assignment 16: SBERT and Finetuning
Learning Objectives
- Learn about SBERT models
- Learn about the idea of finetuning and apply this procedure to a dataset and model of your choosing
It will help a lot to complete this assignment if you have Google Colab pro. As mentioned early in the class, you can get free usage of Colab pro for being a student (see this link on Canvas for instructions). If you are not able to gain access to Colab pro for any reason, please talk to me, and we’ll see if we can find another way forward.
SBERT
SBERT (Sentence-Embeddings using Siamese BERT-Networks) is a hugely influential paper that forms the basis of many modern approaches for computing similarity between two pieces of text. As we saw in the previous assignment, being able to evaluate the similarity between two pieces of text can be used for tasks such as search and question answering. The main ideas of SBERT are described in day 17.
In addition to the writeup from last class, you can learn about SBERT from the original paper as well as this fantastic video.
Finetuning
We’ve put together a notebook with some starter code that will allow you to finetune a sentence embedding model on a dataset of your choosing. You will be building off of this code to try your own experiment.