Transformers Pipeline

Last Updated : 23 Jul, 2025

The Hugging Face pipeline is an easy-to-use tool that helps people work with advanced transformer models for tasks like language translation, sentiment analysis, or text generation. It takes care of the complicated steps behind the scenes like breaking up the text into tokens, loading the right model, and formatting the results properly. This way, users can focus on what they want to do, without worrying about the technical details. Using powerful AI models becomes much easier and more accessible for everyone.

How the Pipeline Works(Step-by-Step)

transformers_pipeline
Workflow of Transformers Pipeline
  • Raw Text: Input text (e.g., "This course is amazing!").
  • Tokenizer: Converts text into numerical "Input IDs" (machine-readable tokens).
  • Model: Processes these IDs to generate "Logits" (raw scores for predictions).
  • Post-Processing: Converts logits into understandable "Predictions" (e.g., "POSITIVE: 99.89%").

Implementation

1. Import the Pipeline

Python 
from transformers import pipeline

2. Create a Pipeline for a Specific Task

You specify the task (e.g., "ner", "translation", "sentiment-analysis", "fill-mask") and optionally the model.

Python 
ner_pipe = pipeline("ner")
translation_pipe = pipeline("translation_en_to_fr")
sentiment_pipe = pipeline("sentiment-analysis")
fill_mask_pipe = pipeline("fill-mask")

Output

Ner-colab-download
NER DOWNLOADED

3. Feed Input Data

Pass your article or sentences to the pipeline.

Python 
text = "Hugging Face Inc. is based in New York City. Its CEO is Clément Delangue."
ner_results = ner_pipe(text)
sent_results = sentiment_pipe(text)
print(sent_results)

Output

[{'label': 'POSITIVE', 'score': 0.9813228845596313}]

4. Pipeline Internals

  1. The pipeline automatically: Tokenizes the input text. Passes tokens through the selected model.
  2. Decodes and post-processes the output for human readability.
    Output: You get ready-to-use results without manual preprocessing or postprocessing.

Use Cases and Examples

1. Named Entity Recognition (NER)

This code below uses the pipeline function from the transformers library to perform Named Entity Recognition (NER).

  • ner_pipe = pipeline("ner"): Sets up an NER model ready to identify named entities like organizations, locations, or monetary values.
  • result = ner_pipe("Apple is looking at buying U.K. startup for $1 billion"): Feeds the sentence to the NER model.
  • print(result): Shows the identified entities (e.g., "Apple" as an organization, "U.K." as a location, "$1 billion" as money) along with their types and positions in the sentence.
Python 
ner_pipe = pipeline("ner")
result = ner_pipe("Apple is looking at buying U.K. startup for $1 billion")
print(result)

Output

[{'entity': 'B-ORG', 'score': ..., 'index': ..., 'start': 0, 'end': 5, 'word': 'Apple'},

{'entity': 'B-LOC', ... 'word': 'U.K.'}]

  • Use Cases: Information extraction, document tagging, compliance, news analytics.

2. Translation

  • translation_pipe = pipeline("translation_en_to_fr"): Sets up a pre-trained model for English-to-French translation.
  • result = translation_pipe("Hugging Face is creating a tool that democratizes AI."): Translates the given English sentence.
Python 
translation_pipe = pipeline("translation_en_to_fr")
result = translation_pipe("Hugging Face is creating a tool that democratizes AI.")
print(result)

Output

[{'translation_text': "Hugging Face crée un outil qui démocratise l'IA."}]

Use Cases: Multilingual support, content localization, cross-border communication.

3. Sentiment Analysis

  • sentiment_pipe = pipeline("sentiment-analysis"): Sets up a model of sentiment analysis to determine if text expresses positive, negative, or neutral sentiment.
  • result = sentiment_pipe("I love using Hugging Face models!"): Analyzes the sentiment of the input sentence.
Python 
sentiment_pipe = pipeline("sentiment-analysis")
result = sentiment_pipe("I love using Hugging Face models!")
print(result)

Output

[{'label': 'POSITIVE', 'score': 0.9998}]

  • Use Cases: Customer feedback, social media monitoring, brand analysis.

4. Fill-Mask (Masked Language Modeling)

  • fill_mask_pipe = pipeline("fill-mask"): Sets up a model to fill in a "masked" (missing) word.
  • result = fill_mask_pipe("Hugging Face is creating a <mask> that democratizes AI."): Provides the sentence with a <mask> token.
  • print(result): Shows the most probable words that fit in the masked position, along with their scores in masked language modelling.
Python 
fill_mask_pipe = pipeline("fill-mask")
result = fill_mask_pipe("Hugging Face is creating a <mask> that democratizes AI.")
print(result)

Output

[{'sequence': 'Hugging Face is creating a tool that democratizes AI.', 'score': ...}, ...]

  • Use Cases: Text autocompletion, grammar correction, language understanding.
Comment

Explore