Zero-Shot Classification

Another core capability provided by ralf is zero-shot classification (ZSC) using Large Language Models (LLMs). Zero-shot classification allows you to classify text inputs without training (or fine-tuning) a model on the specific classification task. Instead, zero-shot classification with LLMs relies on the the in-context learning capabilities of language models to perform classification with few or no examples specified in the prompt.

In ralf, zero-shot classification involves a two-step process. The first uses a causal langauge model (i.e., one that creates text) to generate a class label given an input. Because there may be variations in what this model outputs, we employ a second step to project the class label predicted by the causal LLM onto the closed set of class names. This second step is performed by using a masked language model (i.e., one that embeds text) to compute embeddings for the emitted class name and the reference class names, then selecting the reference name with greatest cosine similarity to the one emitted by the causal LLM.

To perform ZSC, we start by creating the classifier itself:

from ralf import classifier

sentiment_classifier = classifier.ZeroShotClassifier()

We might also want to further specify how the classifier works by configuring attributes of the completion model and/or the encoder model.

sentiment_classifier = classifier.ZeroShotClassifier(
    completion_model_config={'model':'gpt-4'},
    encoder_model_name='all-MiniLM-L6-v2',
)

Next, we add classes to the classifier. We do so by providing a name for each class. Keep in mind that because we are using a language model to perform classification, the class names will have a large impact on the behavior of the classifier. We can optionally provide examples for each of the classes we add.

In the below example, we create a 3-class classifier for sentiment analysis.

sentiment_classifier.add_class("positive", examples=["this couldn't be better"])
sentiment_classifier.add_class("negative", examples=["I'm unimpressed.",
                                                     "not her best work..."])
sentiment_classifier.add_class("neutral")

Just like that, we are ready to test our classifier on new inputs!

text_inputs = ["It's fine, just don't expect too much!",
               "A big miss.",
               "Marvelous performance"]

for x in text_inputs:
    pred, scores = sentiment_classifier(x)
    print(f"'{x}' -> {pred}")

'It's fine, just don't expect too much!' -> neutral
'A big miss.' -> negative
'Marvelous performance' -> positive

As you can see, setting up a zero-shot text classifier in ralf is very straightforward.