This section maps directly to AI-102 exam objective 3.2 — Implement object detection, OCR, and facial recognition solutions. These three capabilities are the workhorses of modern computer vision, enabling systems to interpret images and video the way humans do, only faster and at a scale no human could match. For anyone studying AI-102, understanding how to implement these services means you can build applications that automatically identify products on a shelf, extract text from scanned documents, or verify someone's identity from a photo — all using pre-built Azure AI services.
Jump to a section
A simple way to picture Computer Vision: Object Detection, OCR, and Face Services
Have you ever watched the baggage carousel at an airport and tried to spot your own black suitcase among a hundred others that look almost identical?
That frustrating search is exactly what Computer Vision object detection does, but on a massive scale and at lightning speed. When your suitcase finally appears, you don't just see a black blob — you recognise its specific shape, the bright red ribbon you tied to the handle, and the scuff mark on the side. That's object detection: the AI is trained to pick out specific classes of things (suitcases, people, dogs) from a cluttered scene and draw a box around each one.
Now imagine you're at a foreign airport and need to read a sign in a language you don't understand. You snap a photo, and an app instantly translates the text. That's OCR (Optical Character Recognition) — the AI extracts the written characters from the image, turning a picture of words into editable, searchable text.
Finally, think about how your phone unlocks when it sees your face. It doesn't just detect a face anywhere in the frame — it identifies that it's specifically your face. That's facial recognition, or face services in Azure. It detects the face first, then compares its unique features to a database of known faces to find a match. Together, these three capabilities — detecting objects, reading text, and recognising faces — are the core of what makes computers truly "see" the world around them.
Computer vision is the field of artificial intelligence that trains computers to interpret and understand the visual world. Instead of just storing an image as a grid of pixels, computer vision systems analyse those pixels to extract meaning. The three core capabilities covered in this chapter — object detection, Optical Character Recognition (OCR), and face services — are the most commonly implemented vision features in real-world applications.
Object detection is the ability to find and identify multiple objects within a single image or video frame. Unlike image classification, which answers the question "what is in this picture?" (for example "a cat"), object detection answers "where are the cats and how many are there?" It draws a bounding box around each detected object and assigns a label. Azure provides a pre-built Azure AI Custom Vision service for this, plus the Azure AI Vision Image Analysis API. The underlying technology uses deep learning models, specifically a type of neural network called a Convolutional Neural Network (CNN). CNNs are designed to recognise patterns in visual data by scanning the image in small squares (filters) and learning what combinations of edges, textures, and shapes correspond to each object. When you train a model, you feed it thousands of labelled images — each image has the objects of interest marked with boxes and labels. The model learns the patterns until it can generalise to new, unseen images.
Optical Character Recognition, or OCR, is the process of converting images of text into machine-readable text. This is not the same as scanning a barcode; OCR deals with human-readable characters, whether printed or handwritten. Azure AI Document Intelligence (formerly known as Form Recogniser) is the dedicated service for this, though the Azure AI Vision API also includes OCR capabilities. The pipeline works in stages: first, the image is preprocessed to correct lighting and perspective (skew correction). Next, the system detects regions that contain text — this is called text detection. Then, for each region, the system segments the text into individual lines and characters. Finally, a recognition model interprets each character based on its shape, returning the text string along with confidence scores. OCR replaced the tedious manual data entry process where humans would type out information from printed forms or invoices.
Face services are a specialised subset of object detection focused entirely on human faces. Azure provides three main face capabilities: face detection, face identification, and face verification. Face detection simply finds and locates human faces in an image, returning a bounding box plus optional attributes like age, emotion, or facial hair. Face identification goes further: it compares a detected face against a Face ID database (called a Person Group in Azure) to find a specific person's identity. Face verification asks "are these two faces the same person?" and returns a confidence score. The underlying technology involves extracting a face feature vector — a string of numbers that uniquely represents the geometry of a face (distances between eyes, nose shape, jawline). These vectors are then compared using similarity metrics such as cosine similarity. Azure Face API provides these capabilities as REST endpoints, meaning you can call them from any programming language using HTTP requests.
You can combine these services in powerful ways. For example, a self-checkout system might use object detection to identify products on the belt, OCR to read the expiry date, and face services to verify the cashier's identity before they authorise a price override. Each service returns structured JSON data that your application can process — bounding boxes as coordinates, text as strings, confidence scores as percentages. The key Azure resources you need are a Computer Vision resource (for object detection and OCR) and a Face API resource (for facial recognition), both created through the Azure portal. You then use a client SDK (Software Development Kit) — available in Python, C#, JavaScript, and other languages — to send images or image URLs to these endpoints and receive the analysis results.
What do these services replace? They replace human effort. Manually inspecting thousands of products on a conveyor belt for defects was slow and error-prone; object detection automates it. Keying in data from thousands of invoices was expensive; OCR digitises it instantly. Verifying identity by checking a physical ID card against a person's face was cumbersome; face services can do it in milliseconds. For businesses, this means lower costs, fewer errors, and the ability to scale processing to millions of images. For AI-102 exam candidates, the focus is on knowing which Azure service to use for which task, how to call the APIs, and how to interpret the responses.
Provision the Azure resources
In the Azure portal, create a 'Cognitive Services' resource (multi-service) or separate 'Computer Vision' and 'Face' resources. Note the endpoint URL and subscription key — you will pass these in every API call.
Choose and configure the correct API operation
For object detection, use the Azure AI Vision 'Analyze Image' endpoint with the 'objects' visual feature. For OCR, use the 'Read' endpoint (preferred) or legacy 'OCR' endpoint. For face, use 'Detect' to find faces, 'Identify' to match against a Person Group, or 'Verify' to compare two faces.
Prepare and send the image
Convert your image to a byte array (for local files) or provide a publicly accessible URL. Set the content type to 'application/octet-stream' for binary data or 'application/json' for a URL. Send an HTTP POST request to the chosen endpoint including your subscription key in the 'Ocp-Apim-Subscription-Key' header.
Parse the JSON response
The API returns JSON containing a list of detected objects with bounding boxes (x, y, w, h) and confidence scores. For OCR, you get pages, lines, and words with text strings. For face detection, each face has a faceId and bounding box. Use a JSON parser in your language (e.g., json.loads in Python) to access these fields.
Handle the results and manage limitations
Implement error handling for HTTP 429 (rate limit exceeded) by adding retry logic with exponential backoff. For large documents, check the 'Operation-Location' header in the Read API response and poll the status URL until processing completes. Train your Person Group after adding faces before calling Identify.
A logistics company, FastShip Logistics, receives thousands of parcels each day at its central sorting hub. Each parcel has a shipping label with the recipient's address, barcode, and tracking number. Historically, workers manually read the labels and typed the information into a database — a process that was not only slow but prone to errors like mistyped digits.
The IT team decides to automate the sorting line using three Azure AI services. They set up an overhead camera above each conveyor belt that captures an image of every parcel as it passes. The image is sent to the Azure AI Vision service for OCR. The service reads the shipping label and extracts the tracking number and postcode. This data is then used to automatically route the parcel to the correct outgoing truck lane. For example, if the postcode starts with 'SW', the parcel is routed to the London South-West lane.
But there's a complication: some parcels are irregularly shaped, and the labels might be partially obscured by tape or placed on a curved surface. To handle this, the team also uses object detection. The Azure AI Vision custom detection model has been trained to detect 'Label Area' on any surface. It first finds the region of the parcel that contains the label, then crops that region before sending it to OCR. This two-step process dramatically improves accuracy because the OCR engine only receives the relevant portion of the image, not the entire parcel.
Later, the company wants to add security. Employees must log in to sensitive areas of the warehouse. Instead of using swipe cards or keypads, which can be shared or stolen, FastShip implements facial recognition using Azure Face API. Each employee's photo is enrolled into a Person Group. At the secure door, a camera captures the person's face. The Face API detects the face, then identifies it against the Person Group. If a match is found with high confidence, the door unlocks. If no match is found, an alert is sent to the security team.
The IT team follows these steps:
Create two Azure resources: Azure AI services (multi-service) for object detection and OCR, and a Face API resource for facial recognition.
Train an object detection model using Azure Custom Vision by uploading 200 images of parcels with manually labelled label areas.
Write a Python script that uses the Azure AI Vision SDK to call the Read API (for OCR) and the Custom Vision prediction endpoint (for object detection).
Create a Person Group in Face API and upload employee photos with unique person IDs.
Set up a continuous integration pipeline to retrain the object detection model weekly as new parcel types arrive.
The result: FastShip processes 50,000 parcels per day with 99.5% OCR accuracy, reduces human data entry errors to near zero, and secures its warehouse with frictionless face-based access control. For the IT professional, the day-to-day work involves monitoring the API error rates, retraining models when accuracy drops, and handling edge cases such as low-light images or damaged labels.
The AI-102 exam tests your ability to implement object detection, OCR, and face services using Azure AI. The exam questions are not about memorising deep learning theory; they focus on practical decisions: which API endpoint to call, which SDK method to use, and how to handle the JSON response.
For object detection, the exam loves to test the difference between image classification and object detection. A typical question shows a scenario: 'A company needs to count the number of cars in a parking lot image. Which Azure AI service should they use?' The correct answer is Azure AI Vision Image Analysis with object detection, not image classification, because object detection provides bounding boxes and counts. The trap is that image classification can tell you if a car is present, but it cannot count multiple instances or provide their locations.
Another frequent angle is about OCR endpoints. The exam expects you to know the difference between the Read API (for printed text and handwriting, asynchronous, supports large documents) and the OCR API (legacy, synchronous, for printed text only). If the question mentions a scenario with handwritten notes or multi-page documents, the correct answer is the Read API. If the question says 'real-time, single-page, printed text,' the OCR API could work, but the Read API is still the recommended modern option. The trick is that Microsoft updated its documentation, and newer exam questions reflect that the Read API is the preferred choice for most OCR tasks.
Face services questions are particularly tricky. The exam differentiates between face detection, face identification, and face verification. A common trap question: 'A company wants to unlock a door when an employee's face is recognised. Which operation should they use?' The correct answer is face identification, because it matches a detected face against a known database (Person Group). Face verification would be used if they already know the employee's identity and just want to confirm the face matches a specific enrolled photo. The exam also tests understanding of the Person Group — a container for persons, each of whom can have multiple face images. You must know that Person Groups have a limit of 10,000 persons per group (for the Free tier) and that you must train the group after adding persons before you can call identification.
Key concepts to memorise:
The bounding box format: left, top, width, height (or x, y, w, h) in pixels relative to the image.
Confidence scores: a number between 0 and 1 (or 0 and 100 depending on the API version). Thresholds should typically be set to 0.5 or higher to filter low-confidence results.
The difference between Azure AI Vision and Azure Custom Vision: Azure AI Vision is pre-built for common scenarios (tags, objects, faces, OCR); Azure Custom Vision lets you train your own model with your own labelled images for domain-specific objects.
Face attributes: the Face API can optionally return attributes like age, emotion (happiness, sadness, anger), glasses, and facial hair. These are gated behind a responsible AI policy in some regions.
Exam traps to watch for: - 'Which endpoint should be called to extract text from a scanned PDF of a contract?' The answer is the Read API, not the OCR API, because it supports asynchronous processing of multi-page documents. - 'What is returned by the Detect method of the Face API?' It returns a faceId, a bounding box, and optionally attributes — but it does NOT return a person's name. Names come from identification, not detection. - 'Can you use the Face API to identify a person from a live video stream?' Yes, but you must extract frames and send them to the API — the Face API itself does not process video natively. The exam expects you to know that you need to use a separate video processing solution like Azure Video Indexer or a custom solution.
Finally, the exam tests your understanding of responsible AI. Microsoft has retired certain face capabilities, like emotion recognition from unconsented images in some jurisdictions. A question might ask: 'A company wants to analyse customer emotions in a retail store using surveillance footage. Which Azure service should they use, and what restriction applies?' The correct answer is that emotion recognition is restricted and should not be used without explicit consent in many regions. Microsoft's Responsible AI standard means you must check the latest documentation for geographical and ethical constraints.
Object detection returns bounding boxes and labels for each object found, enabling counting and spatial analysis, unlike image classification which only assigns a single label to the whole image.
Azure Read API is the modern, preferred OCR solution for both printed and handwritten text, supporting asynchronous processing of multi-page documents.
Face detection finds faces and returns faceIds; face identification matches a faceId against a Person Group to return a person's name; face verification compares two faceIds to confirm if they are the same person.
Custom Vision lets you train your own object detection model when the pre-built Azure AI Vision model does not include the objects you need to detect.
Person Groups in Face API require explicit training (Train call) after adding persons before identification requests will work.
Always check the confidence threshold (commonly 0.5 or higher) when filtering results to avoid low-quality detections in production systems.
These come up on the exam all the time. Here's how to tell them apart.
Image Classification
Assigns a single label to the entire image (e.g., 'cat').
Does not provide location information.
Cannot count multiple objects of the same type.
Object Detection
Detects multiple objects and assigns labels to each.
Returns bounding box coordinates for each object.
Can count distinct objects (e.g., '3 cats').
Read API (modern OCR)
Supports printed and handwritten text.
Processes multi-page documents asynchronously.
Returns structured results with pages, lines, and words.
Legacy OCR API
Only supports printed text.
Synchronous, single image only.
Returns simpler text output without page structure.
Face Identification
Matches a face against a database of many known faces.
Returns the identity (person name) of the best match.
Requires a trained Person Group with multiple enrolled faces.
Face Verification
Compares two specific face images to see if they are the same person.
Returns a confidence score and a boolean 'isIdentical' flag.
Does not need a Person Group; just two faceIds.
Azure AI Vision (pre-built)
No training required; works out of the box.
Limited to ~10,000 common object categories.
Cannot be fine-tuned on domain-specific objects.
Azure Custom Vision
Requires uploading and labelling your own training images.
Can detect any object you provide images for.
You can export the model for offline use (e.g., on IoT devices).
Mistake
Object detection and image classification are the same thing.
Correct
Object detection identifies multiple objects in an image and tells you where they are (bounding box). Image classification only tells you what the dominant object or scene is in the whole image without locations.
Beginners often think any AI that identifies a cat in a photo is doing object detection, but the exam specifically tests the distinction of spatial awareness.
Mistake
OCR can read any handwriting perfectly.
Correct
OCR (especially the Read API) can handle handwriting but accuracy depends on legibility, contrast, and the specific handwriting style. It works best on printed text; handwriting accuracy varies widely.
Marketing materials often show flawless handwriting recognition, so beginners expect perfection, but real-world performance is lower.
Mistake
Face identification and face verification are the same operation.
Correct
Face identification matches a detected face against an entire database (Person Group) to find who it is. Face verification compares two specific face images to confirm if they are the same person.
Both operations answer 'is this the same person?' but identification assumes you have a large gallery, while verification assumes you have a claimed identity to check against.
Mistake
You can send a video file directly to the Face API to detect faces.
Correct
The Face API works on static images, not video files. To process video, you must extract individual frames and send them as images. Azure Video Indexer is a separate service for video analysis.
Beginners assume APIs are 'magic' and can handle any format, but the documentation clearly states image input is required.
Mistake
A high confidence score (e.g., 95%) means the result is always correct.
Correct
Confidence scores indicate the model's certainty, not correctness. A model can be confidently wrong if the training data was biased or the input is unusual.
People treat 95% as 'practically 100%', but any probabilistic system has failure modes, especially on edge cases.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Azure AI Vision is pre-trained and can detect common objects like cars, dogs, and furniture out of the box. Azure Custom Vision lets you upload your own labelled images to train a model for specific objects unique to your business, like a specific brand of cereal box.
No, the Face API does not include a built-in celebrity recognition database. You must enroll the faces you want to recognise into a Person Group yourself. Microsoft retired the PersonGroup celebrity feature in 2020.
A bounding box is typically returned as an object with 'x', 'y', 'width', 'height' (in pixels) indicating the top-left corner and size of the box around the detected object.
Check the image quality: low resolution, poor lighting, skewed angles, or blur can reduce accuracy. Use the Read API instead of the legacy OCR API if you need handwriting support. Also ensure you set the correct language parameter.
Only if you use Azure Custom Vision with your own dataset. If you use the pre-built Azure AI Vision 'Analyze Image' API, no training is needed — it works immediately for the 10,000+ common object categories it supports.
Standard tier typically allows 10-20 transactions per second (TPS). If you exceed this, you get HTTP 429 errors. Implement retry logic with exponential backoff, or request a quota increase from Azure Support.
You've finished Computer Vision: Object Detection, OCR, and Face Services. Continue through the AI-102 study guide to build a complete picture of the exam.
Done with this chapter?