Image Classification Tasks

Image classification is a supervised machine learning task where a model learns to assign a label (class) to an entire image from a predefined set of labels. For example, given a picture of a dog, the model outputs 'dog' from classes like 'dog', 'cat', 'bird', etc. The model is trained on a dataset of labeled images, learning to map pixel patterns to labels.

Modern image classification uses deep convolutional neural networks (CNNs). The process:

Training involves: - Dataset: Thousands to millions of labeled images. - Loss function: Categorical cross-entropy for single-label, binary cross-entropy for multi-label. - Optimizer: Stochastic gradient descent (SGD) or Adam to minimize loss. - Backpropagation: Gradients are propagated through the network to update weights. - Epochs: Full passes over the training data. Typical values: 10–100. - Batch size: Number of images processed before updating weights. Common: 32, 64, 128.

Azure offers pre-trained models for common image classification tasks: - Computer Vision API: Can classify images into thousands of categories from the ImageNet dataset. Returns tags with confidence scores. Example: https://<endpoint>/vision/v3.2/analyze?visualFeatures=Categories. - Custom Vision: Allows you to train custom classifiers with your own images. Supports both single-label and multi-label, and also object detection. It uses a transfer learning approach: starts with a pre-trained model (e.g., ResNet) and fine-tunes on your data.

Custom Vision is a no-code/low-code platform for building custom image classifiers. Key concepts: - Project: Contains your model. You specify domain (e.g., 'General', 'Food', 'Landmarks'). - Tags: Labels for your classes. - Training: Upload images, assign tags, and train. You can choose between 'Quick Training' (fast, less accurate) and 'Advanced Training' (more time, higher accuracy). - Prediction: After training, you get an endpoint URL and prediction key. Send images to classify. - Performance metrics: Precision, recall, and mean average precision (mAP). The exam tests understanding of these metrics.

To list Custom Vision projects:

az cognitiveservices account list --resource-group <rg> --query "[?kind=='CustomVision.Prediction']"

To get prediction endpoint:

az cognitiveservices account show --name <name> --resource-group <rg> --query "properties.endpoint"

from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials

endpoint = "https://<your-endpoint>.cognitiveservices.azure.com/"
key = "<your-key>"

client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(key))

with open("image.jpg", "rb") as image:
    result = client.analyze_image_in_stream(image, visual_features=["Categories", "Tags"])

for category in result.categories:
    print(f"Category: {category.name}, Score: {category.score:.2f}")

Image classification is a core computer vision task. Azure provides both pre-built and custom models. The exam focuses on understanding the difference between single-label and multi-label, the capabilities of Computer Vision API and Custom Vision, and key metrics like precision, recall, and mAP. Be prepared to identify scenarios where each service is appropriate and to interpret classification results.

A large e-commerce company needs to automatically categorize product images into categories like 'Electronics', 'Clothing', 'Home & Garden', etc. They have millions of product images from sellers. Using Azure Custom Vision, they create a multi-class classifier with 50 categories. They train on 1,000 images per category. The model achieves 95% precision and 92% recall. They deploy the prediction endpoint as a web service called from their product upload pipeline. When a seller uploads an image, it is classified, and the product is assigned to a category. Misclassifications are flagged for manual review. Performance considerations: latency must be under 500ms; they use Azure App Service with autoscaling. Common issues: sellers upload images with watermarks or white backgrounds that differ from training data, causing misclassifications. They mitigate by adding augmented images during training (rotations, brightness changes) and setting a high probability threshold (0.8) to reduce false positives.

A hospital uses image classification to flag chest X-rays for potential pneumonia. They use a binary classifier: 'Normal' vs 'Pneumonia'. They train on 10,000 labeled X-rays. The model must have very high recall (low false negatives) because missing a pneumonia case is dangerous. They set the probability threshold low (0.3) to catch more positives, accepting more false positives that radiologists review. They use Custom Vision with the 'General' domain. The model is deployed in a HIPAA-compliant environment. They monitor performance over time as new X-ray machines produce different image characteristics. They retrain monthly with new data. Misconfiguration: initially they used single-label but some X-rays show both pneumonia and other conditions; they switched to multi-label to capture all findings.

A social media platform uses image classification to detect inappropriate content like violence, nudity, or hate symbols. They use Azure Computer Vision's pre-built 'Adult' classification (which detects adult content) and combine it with a custom classifier for specific banned symbols. The pre-built model returns scores for 'adult', 'racy', and 'gory' categories. They set thresholds: if 'adult' score > 0.7, automatically block; if between 0.4 and 0.7, send for human review. The custom model is trained on images of banned symbols using Custom Vision. They process millions of images daily, requiring high throughput. They use Azure Functions triggered by blob storage events to classify images asynchronously. Common failure: the pre-built model may flag artistic nudes as adult, causing false positives; they fine-tune thresholds based on user feedback.