How Malachyte Uses Real-Time AI to Tackle Retail's Cold-Start Problem

Turning Shopper Behavior Into Real-Time Recommendations

Online shoppers increasingly expect ecommerce experiences to feel personalized and relevant. For retailers, however, delivering useful recommendations can be difficult when very little is known about a visitor.

This is known as the cold-start problem.

A first-time shopper may have little or no purchase history, browsing history, or established profile that a traditional recommendation system can use.

Malachyte, an AI-powered ecommerce recommendation platform, approached this challenge by focusing on something that is immediately available: the sequence of actions a shopper takes during a session.

The company's approach draws on attention mechanisms used by large language models. Instead of analyzing sequences of words, Malachyte applies similar concepts to sequences of customer interactions with an ecommerce website or application.

A search, click, or other interaction becomes another signal that helps the system understand the shopper and predict what they may want next.

According to Google Cloud, Malachyte's technology has helped some of its retail customers double and sometimes even triple their sales.

The approach depends not only on the recommendation model itself, but also on infrastructure capable of updating shopper context in real time.

Applying Attention-Based AI to Ecommerce

Malachyte's key insight was that neural networks with attention mechanisms could be applied to retail search and product recommendations.

Large language models use attention to understand relationships between items in a sequence. In language, those items can be words or syllables.

For an ecommerce application, Malachyte instead looks at the sequence of customer interactions.

When a visitor arrives, Malachyte creates a vector representing what is known about that user.

For a first-time visitor, there may be very little information available. As the shopper searches, clicks, and interacts with products, the vector is continuously refined.

Each new interaction serves two purposes.

First, it helps the system predict what the shopper may want next. Second, it provides additional information that improves the system's understanding of that shopper.

Malachyte updates the user vector and prediction simultaneously, allowing the recommendation experience to evolve during the session.

The company combines its attention-based neural networks with user-profile updates at approximately 100 milliseconds at a time.

This enables recommendations to adapt while the shopper is still browsing.

Moving Beyond Traditional Recommendation Systems

The idea of using behavioral sequences for recommendations differs from traditional collaborative filtering approaches.

Retailers have historically used collaborative filtering to identify relationships between users and products. These systems can require substantial amounts of interaction history and may rely on information such as third-party cookie-based profiles and demographics.

That creates a challenge when a shopper is relatively unknown.

Malachyte instead focuses on the sequence of interactions within the current session.

By examining what a shopper does during a visit, the system can begin building context without waiting for a large amount of historical information to accumulate.

The approach can also reduce reliance on long-term cookie data, which can provide additional privacy benefits for shoppers.

Malachyte's model uses multimodal vectors to encode information it knows about a user, including browser data, click history, and searches.

The same model can be applied across multiple ecommerce experiences, including:

• On-site search

• Product pages

• Category pages

• Add-to-cart recommendation carousels

This creates a common model structure that can support different parts of the shopping journey.

Connecting Users and Products in the Same Vector Space

Malachyte's system does not only represent shoppers as vectors.

Products in the retailer's catalog are also embedded into the same vector space.

As a shopper interacts with an ecommerce site, the user vector changes. It can move closer to products that become more relevant to the shopper and farther from products that appear less relevant.

This provides the foundation for continuously updating recommendations.

The neural network responsible for computing these embeddings is continuously trained across retailers working with Malachyte.

The company uses anonymized user data to improve the model, meaning activity from shoppers across participating retailers can contribute to improving the overall recommendation system.

Google Cloud describes the resulting architecture as effectively becoming a data cooperative, where each retailer's user activity can help make the model smarter for everyone.

The important distinction is that the system uses the sequence of shopper behavior to continually refine context rather than relying entirely on a long-established customer profile.

Building the Real-Time AI Infrastructure

The recommendation model is only one part of Malachyte's architecture.

To deliver this experience in real time, the company built its system on Google Cloud infrastructure that can continuously process behavioral events and update shopper context.

Two important components are Managed Service for Apache Kafka and Bigtable.

Every behavioral event can stream into a Managed Service for Apache Kafka cluster.

Rather than waiting for a future training job, each event immediately becomes an update to the user's profile in Bigtable.

Bigtable allows Malachyte's services to look up and update the appropriate user vectors.

According to the Google Cloud case study, Kafka and Bigtable operate at approximately 10 milliseconds per step, allowing the overall recommendation loop to operate within the targeted 100-millisecond experience.

This means the system can respond to shopper behavior while the session is still underway.

The architecture is therefore designed around continuous context updates rather than periodic batch processing.

Keeping Product Data Current

Shopper behavior is only one source of information required for accurate recommendations.

Product information also changes continuously.

Malachyte receives product catalog updates, inventory signals, and retailer-specific dimensional data through Cloud Pub/Sub.

Google Cloud says Pub/Sub provides globally accessible REST APIs that allow retailers to establish connections without requiring deep integration work.

This creates another real-time data path alongside shopper behavioral signals.

While Managed Service for Apache Kafka handles behavioral events, Cloud Pub/Sub supports the flow of external product and retailer data into the system.

This helps ensure that the recommendation environment has current information about the products available to shoppers.

Malachyte's agents run on Google Kubernetes Engine (GKE), while model inference runs on Google Compute Engine (GCE).

Together, these components support the AI recommendation architecture.

Why Real-Time Infrastructure Matters for Production AI

Malachyte's architecture highlights an important point about putting AI into production.

The model itself is only one part of the system.

A real-time AI application also needs infrastructure capable of continuously moving data, maintaining context, updating user representations, and making that context available to the model quickly.

In Malachyte's architecture, different Google Cloud services support different parts of that process.

Managed Service for Apache Kafka handles behavioral event streaming.

Bigtable provides the fast key-value store used for user vectors.

Cloud Pub/Sub handles external product and retailer data.

Google Kubernetes Engine runs Malachyte's agents.

Google Compute Engine supports model inference.

Together, these components form the infrastructure surrounding the recommendation model.

This is important because production AI is not simply a question of GPUs and storage.

It also requires systems capable of continuously transforming changing information into usable model context.

What Malachyte's Approach Means for Retail AI

The cold-start problem exists because recommendation systems often have limited information about new users.

Malachyte's approach provides an alternative way to address that limitation by making the current session itself a source of intelligence.

A search provides a signal.

A click provides additional context.

A product interaction changes the user representation.

The next recommendation can then reflect the updated information.

This makes personalization an evolving process throughout the shopping session.

The approach also demonstrates how concepts associated with foundation models and large language models can be applied to other sequential data problems.

In language models, the sequence consists of words and other language tokens.

In Malachyte's system, the sequence consists of shopper interactions.

The underlying principle is similar: understanding what comes next depends partly on understanding what came before.

The Business Impact of Real-Time Personalization

The technical architecture ultimately supports a business objective: helping retailers deliver more relevant experiences to shoppers.

According to the Google Cloud source, Malachyte has helped some retailers double and sometimes even triple their sales.

The reported results illustrate the potential impact of applying real-time AI to ecommerce personalization.

Rather than waiting for extensive customer histories, retailers can use signals generated during the current session.

This can make recommendations more responsive while also reducing the need to rely heavily on long-term cookie-based profiles.

The architecture therefore connects three important elements:

Real-time shopper behavior

→ Continuously updated AI context

→ Personalized recommendations

Production AI Requires More Than Models

Malachyte's migration to Google Cloud's AI architecture demonstrates a broader lesson about production AI.

Deploying AI successfully requires more than a capable model, GPUs, or storage.

It also requires infrastructure that can continuously ingest events, maintain context, update data, deliver product information, and serve model predictions quickly enough to support an interactive experience.

For Malachyte, that foundation includes:

• Managed Service for Apache Kafka for behavioral streaming

• Bigtable for user-vector context

• Cloud Pub/Sub for product and retailer data

• Google Kubernetes Engine for agents

• Google Compute Engine for model inference

The architecture connects these components into a real-time system designed to continuously incorporate changing shopper context.

Conclusion

Malachyte's approach to ecommerce recommendations is built around a straightforward idea: a shopper's current behavior can provide valuable context even when very little is known about that shopper.

By applying attention-based neural networks to sequences of customer interactions, Malachyte continuously updates user vectors and uses that evolving context to personalize search and product recommendations.

Google Cloud infrastructure provides the real-time foundation behind the system.

Managed Service for Apache Kafka streams behavioral events. Bigtable maintains user-vector context. Cloud Pub/Sub delivers product and retailer data. Google Kubernetes Engine runs Malachyte's agents, while Google Compute Engine supports model inference.

The architecture is designed to operate at approximately 100 milliseconds, with Kafka and Bigtable operating at roughly 10 milliseconds per step, according to the Google Cloud case study.

The reported business impact is also significant. Google Cloud says Malachyte has helped some retailers double and sometimes triple their sales.

But the broader lesson goes beyond ecommerce recommendations.

Production AI requires more than models, GPUs, and storage. It needs the real-time streaming, storage, messaging, context management, and serving infrastructure that allows AI systems to respond continuously to changing information.

For retailers facing the cold-start problem, useful personalization may not require years of customer history.

Sometimes, the most valuable signal is the click that happens next.