Implementing Custom Vision Services Using Azure’s AI Services (AI)

Microsoft Azure’s AI Services add smart features to your apps, like seeing, hearing, and translating. A cool tool in this line is Custom Vision, which helps you make your own image recognition systems. This article focuses on Azure AI Services and Custom Vision, explaining how to use them with real-life examples, step-by-step instructions using C#.

Azure AI Services

Azure AI Services has various machine learning algorithms that are developed to address multiple AI tasks effortlessly via simple API requests. Custom Vision is one such tool that provides specialized features for building, deploying, and enhancing custom image classifiers. It allows users to train their models to recognize specific images, which is highly adaptable when developing personalized image recognition projects.

Azure Custom Vision — Use Cases

Custom Vision service, which is one of the Azure AI services, can be applied across a broad range of industries to build your own image identifier model.

• Retail: Automate furniture inventory by identifying products on shelves.
• Manufacturing: Detect defects in products that are passing through the assembly lines.
• Agriculture: Monitor crop health and pest activity.
• Healthcare: Assist in diagnosing diseases through medical imaging.

Getting Started

To start using Azure Custom Vision, you first need to set up an Azure account

• Sign up or log into your Microsoft Azure portal.
• Go to Create a Resource, select AI + Machine Learning option and click on Custom Vision.

Step 1: Create a Custom Vision Project and Install the Necessary Packages
Installing the Azure AI Services Custom Vision SDK in your project is the first step. Use NuGet in your Visual Studio IDE to install the below-given packages, which provide us with the functions required to communicate to the Custom Vision Service.

Install-Package Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training
Install-Package Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction

Step 2: Initialize the Custom Vision Training Client
Using the Custom Vision training key and endpoint, we can set up the training client. This client will manage all interactions with the Custom Vision service, including project creation and model training.

using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;

var trainingClient = new CustomVisionTrainingClient()
{
    Endpoint = "Your Custom Vision Endpoint",
    ApiKey = "Your training key"
};

// Create a new project
Console.WriteLine("Creating new project:");
var project = trainingClient.CreateProjectAsync("My New Project").Result;

A new project is created in the Custom Vision service. All our models, tags, and images are now present in this newly created project.

Get Chandrahass Tvs’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Step 3: Create a Tag and Upload Images
Tags are labels for the images that the model will learn to recognize. After creating a tag, images are uploaded to the project with their respective tags.

var tag = trainingClient.CreateTagAsync(project.Id, "My Tag").Result;
using (var stream = new FileStream("path_to_image.jpg", FileMode.Open))
{
    trainingClient.CreateImagesFromDataAsync(project.Id, stream, new List<Guid>() { tag.Id }).Wait();
}

Step 4: Training the Custom Vision Model
In this step, we train our custom vision model with the uploaded images. The training process uses these images and their tags to understand how to label the uploaded images. We use the TrainProjectAsync function to make an asynchronous call to the training client to start the training.

var iteration = trainingClient.TrainProjectAsync(project.Id).Result;
Console.WriteLine("Training started...");

Step 5: Publishing the Model
Once training is complete and the model is evaluated, publish the model.

trainingClient.PublishIterationAsync(project.Id, iteration.Id, "modelname", "predictionResourceId").Wait();
Console.WriteLine("Model published!");

Step 6: Test and Predict Using the Published Model
Once we publish the model, we can use it to predict the classification of new images. This step shows how the model can be applied in real-world scenarios.

var predictionClient = new CustomVisionPredictionClient()
{
    Endpoint = "Your Custom Vision Endpoint",
    ApiKey = "Your prediction key"
};

using (var stream = new FileStream("path_to_test_image.jpg", FileMode.Open))
{
    var result = predictionClient.ClassifyImageAsync(project.Id, "modelname", stream).Result;
    foreach (var prediction in result.Predictions)
    {
        Console.WriteLine($"Tag: {prediction.TagName} Probability: {prediction.Probability:P1}");
    }
}

Considerations While Using Custom Vision

When integrating Azure Custom Vision into your applications, there are several critical factors to consider to ensure optimal performance and effective use:

Data Quality and Quantity: The success of the custom vision model depends on the quality and diversity of the images used for the training. Additionally, the quantity of data matters; generally, more data leads to improved model performance.

Model Performance: Monitoring and continuously evaluating the model’s performance is essential to ensuring it meets the application’s requirements. This includes checking for accuracy, precision, recall, and understanding how well the model performs with new data.

Operational Costs: Running AI models in the cloud incurs costs associated with processing power, storage, and API calls. It’s important to optimize the model and its usage to balance performance and cost.

Compliance and Security Considerations

Ensuring compliance and security when implementing Custom Vision Services using Azure’s AI Services is crucial. Key considerations include implementing strong encryption and access controls, adhering to data privacy regulations such as GDPR and HIPAA, utilizing Azure’s network security features, and implementing rigorous monitoring and logging.

Integration with Other Azure Services

Combining Custom Vision with services like Azure Logic Apps for workflow automation, Azure IoT for edge computing, and Azure Machine Learning for advanced AI model management can enhance capabilities and create more comprehensive solutions.

Additional References

1. Azure Custom Vision Documentation

3. Custom Vision SDK for .NET Reference

4. Azure Samples GitHub Repository

5. Microsoft Learn