Azure AI Search (Cognitive Search)

Azure AI Search is a managed cloud service that provides full-text search over indexed content, with optional AI enrichment to extract insights from unstructured data. It is part of the Azure AI portfolio and is used to build search experiences for web and mobile applications, enterprise content management, and e-commerce.

Traditional databases struggle with full-text search at scale—they lack ranking, typo tolerance, and natural language support. Azure AI Search provides: - Full-text search with Lucene-based query engine - AI enrichment via Cognitive Services (e.g., OCR, entity recognition, key phrase extraction) - Knowledge store to persist enriched data for downstream analytics - Semantic search for better understanding of query intent - Scalability up to billions of documents

The core components are: - Index: a collection of documents and their fields, with a schema defining field types (Edm.String, Edm.Int32, etc.) and attributes (searchable, filterable, facetable, sortable). - Data source: a connection to data in Azure Blob Storage, Cosmos DB, SQL Database, or other sources. - Skillset: a set of AI skills (e.g., OCR, language detection) that transform and enrich data during indexing. - Indexer: a pipeline that connects data source to index, optionally applying a skillset. - Query: a request to the search endpoint using REST or SDK, returning ranked results.

An index is defined by a JSON schema. Example:

{
  "name": "hotels",
  "fields": [
    {"name": "HotelId", "type": "Edm.String", "key": true, "searchable": false, "filterable": true},
    {"name": "HotelName", "type": "Edm.String", "searchable": true, "filterable": true, "sortable": true},
    {"name": "Description", "type": "Edm.String", "searchable": true, "analyzer": "en.microsoft"},
    {"name": "Rating", "type": "Edm.Double", "filterable": true, "sortable": true, "facetable": true}
  ]
}

Data sources define where data comes from. Supported types:

An indexer runs on a schedule or on-demand, pulling data from the data source, optionally applying a skillset, and writing to the index. Indexers track changes using high-water mark or change tracking.

Skillsets are a sequence of cognitive skills. Built-in skills include: - OCR: extracts text from images (JPEG, PNG, BMP, TIFF) - Language detection: detects language of text - Entity recognition: extracts entities like people, organizations, locations - Key phrase extraction: identifies key phrases - Sentiment analysis: determines sentiment score (positive/negative/neutral) - PII detection: detects personally identifiable information - Image analysis: describes images, generates tags - Custom skill: allows calling a custom API (e.g., Azure Function) for specialized processing

Skills are defined in a skillset JSON. Each skill has inputs and outputs that map to fields in the index. Example OCR skill:

{
  "@odata.type": "#Microsoft.Skills.Vision.OcrSkill",
  "context": "/document",
  "defaultLanguageCode": "en",
  "inputs": [
    {"name": "image", "source": "/document/normalized_images/*"}
  ],
  "outputs": [
    {"name": "text", "targetName": "extractedText"}
  ]
}

The knowledge store is a feature that persists enriched data to Azure Storage (Blob, Table, or Azure SQL) for analysis or reuse. It is defined as a projection in the skillset. Example:

"knowledgeStore": {
  "storageConnectionString": "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...",
  "projections": [
    {
      "tables": [],
      "objects": [
        {
          "storageContainer": "enricheddata",
          "source": "/document"
        }
      ],
      "files": []
    }
  ]
}

Queries are sent via REST or SDK. Example REST query:

POST /indexes/hotels/docs/search?api-version=2023-10-01-Preview
{
  "search": "luxury hotel",
  "searchMode": "any",
  "filter": "Rating gt 4",
  "orderby": "Rating desc",
  "select": "HotelName, Description, Rating",
  "top": 10
}

Semantic search uses AI to understand query intent. It returns: - Semantic caption: a short, relevant excerpt - Semantic answers: direct answers to questions (e.g., "What is the capital of France?")

Enabled by setting queryType: 'semantic' and semanticConfiguration in the index.

Using Azure CLI:

# Create a search service
az search service create --name mysearch --resource-group myrg --sku basic

# List admin keys
az search admin-key show --service-name mysearch --resource-group myrg

# Create an index
az search index create --service-name mysearch --name hotels --fields @index.json

Using Azure PowerShell:

New-AzSearchService -Name mysearch -ResourceGroupName myrg -Sku Basic
Get-AzSearchAdminKey -ServiceName mysearch -ResourceGroupName myrg

A large online retailer uses Azure AI Search to power product search on their website. They index millions of products from a Cosmos DB database. The index includes fields like product name, description, category, price, and reviews. They use a skillset with entity recognition to extract brand names and key phrase extraction to generate tags. A knowledge store persists enriched data to Blob Storage for analytics. The search service is scaled to S3 with 12 replicas to handle 10,000 QPS during peak sales. Common issues: indexer failures due to schema mismatches (e.g., a field expected as Edm.Double receives a string) or throttling from Cognitive Services API rate limits. They mitigate by using batch size and retry policies.

A healthcare organization indexes medical records stored as PDFs in Azure Blob Storage. They use OCR skill to extract text from scanned documents, language detection to identify documents in multiple languages, and PII detection to redact patient information. The index allows doctors to search for diagnoses, medications, and procedures. They use semantic search to provide direct answers to queries like 'What is the dosage of aspirin?'. The knowledge store stores enriched JSON for compliance audits. Misconfiguration: forgetting to set the OCR skill's 'defaultLanguageCode' leads to poor text extraction for non-English documents.

A law firm indexes millions of legal documents from a SharePoint Online site. They use custom skills to call an Azure Function that performs custom legal entity extraction (e.g., case numbers, court names). The index is used by paralegals to find relevant cases. They use facets for document type and date. They set up a schedule indexer to run nightly. Performance consideration: indexing large documents with many images can be slow; they use the 'imageAction' parameter to generate normalized images. Common problem: indexer fails because the SharePoint data source requires a valid connection string with OAuth; they use managed identity for authentication.