Box Brings Multimodal AI Agents to Enterprise Content With Gemini Embeddings 2

Box Brings Multimodal AI Agents to Enterprise Content With Gemini Embeddings 2

Google Cloud and Box are expanding Box's Agentic Platform with multimodal capabilities powered by Gemini Multimodal Embeddings 2, targeting a longstanding limitation in enterprise AI: much of the information stored in business documents is not just text.

Financial reports contain tables and charts. Clinical material can include images and structured risk information. Legal documents may combine scanned pages, text and visual elements. Presentations can contain diagrams whose meaning depends heavily on their layout.

Google Cloud says the integration is designed to help AI agents work across these different forms of enterprise content by representing text, images, document pages, rendered spreadsheet tables and visual charts within a unified multimodal vector space.

The announcement represents an evolution of traditional retrieval-augmented generation, or RAG, toward systems that can retrieve and reason across multiple content modalities rather than primarily treating documents as collections of text.

Why Text-Only RAG Has a Limitation

Traditional RAG has become an important architecture for enterprise AI.

A typical system retrieves relevant passages from company documents and provides those passages to an AI model as context for generating an answer.

That approach works particularly well when the important information exists in prose.

But enterprise documents are rarely composed entirely of prose.

A financial spreadsheet may communicate meaning through the relationship between row labels, column headers and numerical values.

A process diagram may depend on arrows and spatial positioning.

A presentation may contain a chart that conveys information not fully captured by its surrounding text.

Google Cloud argues that converting these elements into flat text can disconnect information from the visual and spatial relationships that give it meaning.

This creates a challenge for enterprise agents.

An agent may find the right document but still fail to retrieve or interpret the most important piece of information because it exists in a chart, table, image or page layout.

That is where multimodal embeddings enter the architecture.

What Gemini Multimodal Embeddings 2 Adds

Google Cloud describes Gemini Multimodal Embeddings 2 as providing a unified multimodal vector space for different types of content.

The technology can represent:

• Text

• Raster images

• Document pages

• Rendered spreadsheet tables

• Visual charts

within the same semantic representation space.

The goal is to make different content types searchable and relatable through a common representation.

For example, a natural-language query could be used to find a relevant chart inside a large collection of presentation slides.

Similarly, a document page can be embedded while retaining its visual hierarchy instead of first being reduced to arbitrary text chunks.

The Google Cloud post identifies three major capabilities.

Crossmodal retrieval

Text-to-visual and visual-to-text retrieval allows users to search across modalities.

Layout-aware document embedding

Document page renderings can retain visual structures such as callout boxes and hierarchy.

Heterogeneous format bridging

The approach supports content across formats including:

.docx | .xlsx | .pdf | .pptx | .png | .csv

Three Enterprise Agent Patterns

Google Cloud and Box describe three design patterns for using multimodal embeddings in enterprise agents.

They cover financial analysis, clinical decision support and cross-document reconciliation.

1. Financial and Analytical Reporting

Financial documents often combine numerical tables, charts and written explanations.

Text-only indexing can separate numbers from the table structures that explain what those numbers represent.

Google Cloud says multimodal embeddings can preserve the physical relationships between tables and charts.

This can allow financial agents to understand relationships such as a column header applying to a particular set of metrics.

The article also describes visual trend analysis, where agents can compare written summaries against visual trends in accompanying charts.

Another capability is contextual sourcing, allowing users to retrieve the page, table or chart supporting a particular metric.

Why this matters

The broader implication is that enterprise AI does not always need to convert every piece of information into prose before making it searchable.

In some workflows, the structure itself is part of the information.

2. Multimodal Clinical Decision Support

The second pattern focuses on healthcare and clinical environments.

Google Cloud describes scenarios where relevant information may be distributed across physical photographs, pathology imagery and structured triage information.

The multimodal approach is designed to bring these different forms of information into a shared representation.

The source describes three potential uses:

• Cross-modal clinical synthesis

• Identification of visual anomalies

• Risk-aware decision support

For example, the system could connect visual clinical evidence with other structured information rather than treating each source independently.

Importantly, these are use cases described by the Google Cloud and Box announcement. They should not be interpreted as evidence that the system independently provides medical diagnoses or replaces clinical professionals.

3. Cross-Document Multimodal Synthesis

The third pattern addresses a common enterprise problem: important information is distributed across different files.

A business process might involve:

PDF meeting minutes + Excel spreadsheet + PNG image + email

Traditional systems may process these assets separately.

A multimodal approach aims to connect information across them.

Google Cloud and Box describe potential applications including:

• Cross-file synthesis

• Identifying conflicting information

• Visual-to-text auditing

One example given in the announcement involves identifying outdated pricing in an image by comparing it with information in financial spreadsheets. Another involves comparing a signed PDF contract against a legal review email to identify missing clauses or changes.

Box's Agentic Platform Moves Beyond Search

The announcement frames the integration as more than an improvement to document search.

Google Cloud describes Box's Intelligent Content Management platform as a governed content foundation where AI agents can interrogate, cross-reference and act on enterprise content while operating within existing compliance and security controls.

The platform includes AI agents covering functions such as:

• Search

• Metadata extraction

• Research

• Analysis

• Composition

The combination with multimodal embeddings is intended to allow those agents to work across more forms of enterprise information.

This changes the role of enterprise content repositories.

Instead of being primarily places where employees store and retrieve files, they can become sources of structured context for AI-powered workflows.

From Document Search to Content Reasoning

The architectural shift can be understood through a simple progression:

Storage

↓

Search

↓

Text Retrieval

↓

Multimodal Retrieval

↓

Agentic Analysis

↓

Cross-Document Reasoning

The important change is not simply that AI can "see" images.

The more significant architectural development is the ability to represent different content types within a common semantic space and use that representation as part of retrieval.

For enterprises, this could make it easier to build workflows where an agent needs information from multiple document types rather than a single text source.

Why Multimodal Embeddings Matter for Enterprise AI

The announcement points to a broader industry shift toward multimodal-first enterprise AI architectures.

Enterprise data has always been multimodal.

What is changing is the ability to build AI systems that can retrieve and connect those different modalities.

Consider a corporate acquisition project.

Relevant information could exist in:

• A PDF due-diligence report

• An Excel financial model

• A presentation containing market charts

• Scanned legal documents

• Images and supporting materials

A text-only system may extract portions of this information.

A multimodal system aims to preserve more of the relationships between those assets.

That does not automatically make the resulting AI workflow accurate.

The quality of retrieval, embedding, agent reasoning, source grounding and human review remains important.

But it changes what can be represented and retrieved.

What This Means for Enterprise AI

The Box and Google Cloud announcement highlights a broader architectural change.

Enterprise AI systems are moving from models that primarily retrieve passages toward agents that can work across richer representations of business information.

For organizations, several lessons stand out.

Preserve document structure

Flattening every document into text can remove relationships that matter.

Treat images and charts as data

Visual content can contain business-critical information that traditional search systems may overlook.

Connect different file formats

Real workflows rarely operate inside a single document type.

Ground agents in governed enterprise content

AI agents need access to authorized information and appropriate controls when working with sensitive business data.

Benchmark retrieval quality

Multimodal capability alone does not guarantee useful results. Precision and grounding remain important evaluation areas.

The Bigger Picture

The integration of Gemini Multimodal Embeddings 2 into Box's Agentic Platform illustrates where enterprise retrieval systems could be heading.

The first generation of enterprise RAG largely focused on making text searchable and usable by language models.

The next stage is broader.

Text + Images + Tables + Charts + Layout + Cross-Document Relationships

can become part of the retrieval layer that feeds AI agents.

That creates opportunities for workflows that were difficult to support when enterprise information was reduced to isolated text passages.

At the same time, the announcement does not establish that multimodal embeddings solve the broader reliability challenges associated with enterprise AI.

Organizations still need to consider retrieval precision, permissions, data governance, evaluation, hallucination risk and human oversight.

The technology changes the available representation of enterprise information.

How effectively organizations use that representation will depend on the surrounding architecture and controls.

What Happens Next?

The announcement points toward an enterprise AI architecture where documents are no longer treated simply as collections of text.

Tables, charts, images, layouts and cross-document relationships can become part of the information available to AI agents.

For Box, the integration with Gemini Multimodal Embeddings 2 is positioned as a way to extend its Agentic Platform toward this multimodal model of enterprise content management.

For enterprise AI teams, the larger lesson is architectural.

As agents become more capable, the quality of the information layer beneath them becomes increasingly important.

A system that can retrieve only text may miss information embedded in visual and structural elements.

A multimodal retrieval layer can broaden that foundation.

The challenge then moves to ensuring that the retrieved information is relevant, authorized, auditable and correctly interpreted.

That makes multimodal embeddings an important component of the emerging enterprise agent stack, rather than simply another enhancement to document search.