Enterprise Search with Google Cloud

Enterprise search is the practice of making an organization's internal data—documents, emails, databases, intranets, wikis—searchable from a single interface. Unlike web search (which crawls public pages), enterprise search must handle authentication, diverse data formats, and varying access controls. Google Cloud's solution is Vertex AI Search (formerly Enterprise Search on Google Cloud), a managed service that uses Google's search technology (the same that powers Google.com) to index enterprise data and provide fast, relevant results.

Vertex AI Search consists of three core components: data stores, search apps, and indexing pipelines.

To create a search app:

Example API call to search:

curl -X POST -H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
-H "Content-Type: application/json" \
https://discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/global/dataStores/DATA_STORE_ID/servingConfigs/default_search:search \
-d '{
  "query": "cloud security",
  "pageSize": 10
}'

You can also use the Google Cloud console to view indexing status: under the data store, the Documents tab shows the count of indexed documents and any errors.

Vertex AI Search automatically scales to handle millions of documents. The service is regional (choose a region like us-central1) but can serve global queries. Latency is typically under 200 ms for simple queries. For high-throughput applications, you can provision additional serving capacity (purchased through reserved capacity units).

A multinational corporation with 50,000 employees maintains thousands of internal documents across multiple departments. Previously, employees had to search SharePoint, Confluence, and file shares separately. The company deployed Vertex AI Search with a data store connected to Cloud Storage containing PDFs and Word docs. They also indexed their BigQuery table of IT support tickets. The search app was embedded in the company intranet. They configured synonyms for technical jargon (e.g., 'VM' and 'virtual machine') and boosted results from the IT department for queries containing 'password' or 'login'. The solution reduced average search time from 3 minutes to 15 seconds. However, they initially forgot to set up access control, causing users to see documents they shouldn't—a critical misconfiguration. They later integrated with Cloud IAM to enforce document-level permissions.

An online retailer with 2 million products wanted to improve site search. They used Vertex AI Search to index their product catalog stored in BigQuery (fields: product_name, description, price, category). They configured autocomplete with a deny list to exclude inappropriate terms. They also used boosting to prioritize in-stock items and items on sale. The search API was called from their web front-end. During a flash sale, the system handled 10,000 queries per second with 99.9% uptime. The main challenge was handling product variants (e.g., different sizes) — they had to ensure each variant was a separate document with distinct SKUs. A common mistake was not normalizing text (e.g., 'iPhone 12' vs 'iphone12'), which they solved by adding synonyms and using the built-in spell correction.

A law firm needed to search through millions of legal documents for e-discovery. They stored documents in Cloud Storage and used Vertex AI Search with custom metadata fields (case number, date, author). They used boosting to prioritize documents with higher relevance scores from previous searches. However, they hit the 10 MB document size limit for some scanned PDFs. They pre-processed these files by splitting them into smaller parts. They also had to handle OCR for scanned documents—Vertex AI Search automatically extracts text from images, but accuracy varied. They supplemented with manual tagging. The biggest issue was latency: complex queries with many filters took over 2 seconds. They optimized by reducing the number of fields in the schema and using pre-filtering.