Describe features of Natural Language Processing workloads on Azure →mediumMultiple ChoiceObjective-mapped
AI-900 Practice Question: Describe features of Natural Language Processing workloads on Azure
What is tokenization in the context of natural language processing?
⚠ Common exam trap
Watch out — candidates often confuse tokenization with other 'token' concepts in Azure (like SAS tokens or OAuth tokens), leading candidates to pick option C, which is about API security rather than NLP preprocessing.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Breaking text into smaller units (tokens) for processing by language models
Tokenization is the process of breaking text into smaller units called tokens, which can be words, subwords, or characters. This is a fundamental preprocessing step in natural language processing because language models like GPT or BERT operate on discrete tokens rather than raw text. Each token is mapped to an integer ID from a vocabulary, enabling the model to process and generate language mathematically.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Encrypting text for secure transmission to AI services
Why it's wrong here
Encryption is a security mechanism that transforms text into ciphertext using cryptographic keys to protect data confidentiality during transmission or storage; it does not split or segment text for linguistic analysis. Tokenization, by contrast, is a structural preprocessing step for NLP that leaves text legible (or maps it to token IDs) and has nothing to do with securing data.
- ✓
Breaking text into smaller units (tokens) for processing by language models
Why this is correct
Tokenization breaks raw text into smaller, meaning-bearing units known as tokens — commonly words, subwords, or characters using algorithms like WordPiece or Byte-Pair Encoding. These tokens are then mapped to integer IDs in a vocabulary that language models (e.g., BERT, GPT) can process numerically, making this the correct definition of NLP tokenization.
- ✗
Assigning security access tokens to API calls
Why it's wrong here
Security access tokens are credential strings (e.g., OAuth 2.0 bearer tokens) used to authenticate and authorize API requests to Azure AI services. NLP tokenization, in contrast, is the content-level process of dividing text into semantic units (words or subwords) for model input — the shared word 'token' is a coincidental naming overlap, not a functional similarity.
- ✗
Converting text into a numerical score for sentiment analysis
Why it's wrong here
Sentiment scoring is a downstream task: after tokenization and model inference, the model maps the token sequence to a numeric polarity score (e.g., -1 to +1). Tokenization itself is purely the mechanical splitting of raw text into lexical units — it does not quantify sentiment, so this option misidentifies the preprocessing stage as the task it feeds.
Go deeper
Related to this question
Learn chapter
Azure AI Language Service
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Natural language processing
Natural language processing (NLP) is a branch of artificial intelligence that helps computers understand, interpret, and respond to human language in a way that is both meaningful and useful.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.