Google adds BigQuery search tools for unstructured data

Google has added new search and embedding features to BigQuery, including two generally available releases and one public preview feature for unstructured data analysis.

BigQuery now offers Autonomous Embedding Generation and AI.SEARCH in general availability, while Hybrid Search has entered public preview. The additions address a longstanding challenge for companies trying to query and analyse documents, images, audio files, and free text alongside structured records in a single system.

BigQuery has long been used as a data warehouse for structured information, but organisations have typically needed separate tools to extract and search meaning from unstructured material. That has meant moving data outside the warehouse, building separate large language model workflows, and maintaining additional search indexes.

Google aims to keep more of that work inside BigQuery. Users can define a schema column for embeddings, and the system then generates them asynchronously as new data arrives.

That process applies to text records and now to images through ObjectRefs. Users can choose external embedding models, such as Vertex AI text embeddings, or use Gemma embedding models within BigQuery.

Search update

The second generally available release centres on AI.SEARCH, a function that lets users run natural language searches over data that has already been embedded. Google said it has been tuned for single-query execution, which is particularly relevant for online applications and agent-style search tasks.

Google said it has seen gains of up to 133 times in slot efficiency for single-query searches. That suggests lower compute use for the same search workload, an important metric for customers trying to control cloud spending while serving large numbers of user queries.

In practice, the feature is designed to return records that are semantically related to a user's question rather than matching only exact keywords. In Google's example, a researcher searching for treatments for advanced tumours could retrieve clinically related trial records even when the wording differs from the query.

But not every enterprise query depends on broad semantic similarity. Some users need exact retrieval for specialised product names, identifiers, or codes with little meaning outside a narrow field.

Hybrid model

Hybrid Search addresses that need by combining vector-based semantic search with keyword-based lexical matching. Google said the feature uses methods including Reciprocal Rank Fusion and BM25 to balance conceptual relevance with exact text matching.

Google used the example of an immunotherapy drug designation such as MK3475, where a purely semantic system may struggle to rank results properly because the term is alphanumeric and highly specific. Hybrid Search is intended to improve retrieval in those cases by weighing both the keyword itself and the surrounding meaning.

Users can run hybrid queries through AI.SEARCH and VECTOR_SEARCH by setting hybrid mode or lexical search parameters. Google also said vector indexes can be extended to include keyword data so lexical search can run faster at scale.

The changes are part of a broader effort to make unstructured data analysis a direct workload inside BigQuery rather than a separate stack. Documents are a central example, especially in sectors such as healthcare, where companies may manage thousands of PDF files linked to trials, treatments, approvals, and patient information.

Google described a workflow that starts with querying documents in Google Cloud Storage through Object Tables, then processing them with built-in AI functions for parsing, generation, and classification. Search and embeddings sit in what Google describes as the grounding stage, while graph tools and conversational analytics are used to connect entities and query findings.

That architecture reflects a broader trend across the cloud market, as vendors try to combine data storage, retrieval, and AI inference into fewer managed services. The commercial appeal is straightforward: customers want to reduce the engineering work required to maintain separate pipelines for warehouse data, search indexes, and model orchestration.

For BigQuery, the update also strengthens its position against specialist vector databases and standalone retrieval systems that have become common in generative AI projects. By bringing embedding generation and hybrid retrieval into the data warehouse, Google is arguing that customers can build document search and retrieval-augmented systems without exporting data to a separate platform.

Google said the goal is to support an end-to-end platform for unstructured data analytics inside BigQuery, covering access, processing, grounding, relationship mapping, and activation through conversational interfaces. Hybrid Search remains in public preview, while Autonomous Embedding Generation and AI.SEARCH are now generally available.