Computer Vision: Image Analysis and Classification

Computer vision is the branch of artificial intelligence that trains computers to interpret and understand the visual world. Instead of a human manually reviewing every photo or video frame, a computer vision system can automatically detect objects, read text, recognise faces, and even estimate depth and movement. For the AI-102 exam, you focus specifically on image analysis (extracting detailed information from a single image) and image classification (assigning that image to one or more pre-defined categories).

To understand how this works, you first need to know what an image really is to a computer. A digital image is just a grid of tiny squares called pixels. Each pixel has a colour value. For example, a 200x200 pixel image has 40,000 separate pixels. The computer sees these as a massive grid of numbers — not a picture of a cat. The challenge is turning that grid of numbers into a meaningful label like "cat" or "dog."

Machine learning models specially trained for vision tasks use a technique called deep learning. Deep learning models are built from layers of artificial neurons loosely inspired by the human brain. A common architecture for image tasks is the Convolutional Neural Network (CNN). Think of a CNN as a series of filters: the first layer detects simple patterns like edges and colours. The next layer detects slightly more complex shapes like circles or lines. Deeper layers detect full objects like eyes, noses, or wheels. The final layer combines those detections to make a classification decision.

Now consider image analysis versus image classification. Image analysis (often called image understanding) is about extracting rich metadata from an image. For example, you upload a photo of a restaurant, and the service returns a list of detected objects: table, chairs, menu, person, plate, food. It might also extract text from the menu (optical character recognition, or OCR), detect whether the people are happy or sad (sentiment), and estimate the room's lighting conditions. Analysis produces a dense description.

Image classification is narrower. You train a model to assign one or more labels to an entire image. For instance, you have 10,000 photos of different animals, each labelled as "mammal" or "bird." The model learns the visual patterns that separate the two classes. Once trained, you give it a new photo of a penguin, and it outputs the label "bird" with a confidence score of, say, 95%. The confidence score tells you how sure the model is. If the score is low (e.g., 40%), the model is uncertain — this is a signal to reject or flag the result.

In Azure AI services, these capabilities are available through the Azure AI Vision service. You do not need to build your own CNN from scratch. Instead, you call pre-built REST APIs or use the SDK to submit an image URL or binary data. The service handles the complex deep learning inference and returns JSON results. Key operations include:

The pre-built models are trained on massive datasets like ImageNet, which contains millions of labelled images across thousands of categories. This means you can immediately use the service for common tasks without any custom training. However, if your images are specialised — say, X-ray scans or vintage car parts — you may need to train a custom model using Azure Custom Vision or a Custom Neural Network.

Why does this matter? Image analysis and classification replace manual human inspection. A human can look at a photo and know it is a cat, but a human cannot review 10,000 photos per second. Computer vision scales that ability. Businesses use it for inventory management (photos of shelves to count stock), content moderation (flagging explicit images), medical imaging (detecting tumours in X-rays), and autonomous vehicles (identifying pedestrians and traffic signs).

On the AI-102 exam, you will be asked to differentiate between these operations, choose the correct API endpoint, interpret JSON results, and handle error codes. You also need to understand how to optimise image input (e.g., resizing to within 4MB, using supported file types like JPEG, PNG, or BMP) and how to manage rate limits and pricing tiers.

Imagine you work for a large online fashion retailer called TrendLoop. Your company receives 50,000 new product photos every day from suppliers. Each photo shows a clothing item on a white background. Your boss wants to categorise every item into departments, detect the colour, and extract any text on the labels (like size or care instructions). Doing this manually would require a team of 50 people working full time. Instead, you build an automated solution using Azure AI Vision.

Step 1: Set up an Azure subscription and create an Azure AI Vision resource in the Azure Portal. You configure it in the region closest to your data centre (e.g., West Europe). You note the endpoint URL and the API key — both are required to authenticate every request.

Step 2: Write a script that takes each product photo from a shared blob storage container, resizes it so it does not exceed 4MB, and sends it to the Analyse Image API. You use the visual features parameter set to 'Tags', 'Objects', 'Description', and 'Read'. The service returns a JSON object. For example, for a photo of a red dress:

Step 3: Parse the JSON to extract the top tag (e.g., "dress") and map it to your inventory database category "Dresses." Store the colour value and the extracted text into separate fields. This is now an automated pipeline: photos go in, categorised product data comes out.

Step 4: Handle errors and edge cases. If a photo arrives with poor lighting, the analysis might return low-confidence tags. You set a threshold: only accept tags with confidence >= 80%. Lower confidence items are flagged for manual review. You also handle HTTP error codes, like 429 (too many requests), by implementing retry logic with exponential back-off.

Step 5: Monitor performance. After one week, you review the metrics in Azure Monitor. You find that 93% of items are classified correctly, but the model confuses "blouses" and "shirts" 7% of the time. You decide to train a custom model using Azure Custom Vision with 200 labelled examples of each category. This improves accuracy to 98%.

As an IT professional, you never train the deep learning model from scratch. You configure pre-built services, manage authentication, handle data ingestion, and design fallback logic. You also decide when to use the pre-built model versus when to invest in custom training. The exam tests your ability to make these exact decisions.