How Zoho Is Quietly Redefining Enterprise AI With Zia LLM

Zoho has just made its SaaS suite smarter, not merely wiser.

The enterprise tech major has designed an ambitious artificial intelligence (AI) suite based on its in-house large language model (LLM), Zia, which includes a no-code agent builder, customisable AI assistants, and automatic speech recognition (ASR) models for English and Hindi.

“It will help businesses we serve more seamlessly integrated and function more efficiently,” CEO Mani Vembu claimed after rolling out the stack at Zoholics India, the company’s annual user conference, in Bengaluru in July.

Most enterprises are content integrating third-party foundation models, or relying on open-source alternatives. Zoho took the road less travelled by building its own family of small, domain-specific LLMs entirely in-house.

“It’s not just about our technological prowess, it signals our long-standing playbook of vertical integration, tight control over data and infrastructure, and a deliberate focus on cost-efficiency and privacy,” Vembu said.

Zoho has tailored the LLM model for specific business use cases. Built on NVIDIA’s accelerated computing platform, Zia LLM powers use cases like data extraction, summarisation, code generation, and task automation.

The Zia family consists of compact, domain-specific models, trained between 1.3 Bn and 7 Bn parameters, and embedded deep inside Zoho’s enterprise SaaS stack.

“We felt that in certain use cases, we could be more efficient,” said Vembu. “We trained each of these models – 1.3 Bn, 2.6 Bn, and 7 Bn – for specific scenarios. This allows us to choose the right model for each task, which brings significant efficiency benefits.”

An Extension Of Zoho, Not A Pivot

Zoho is not tempted to tag itself AI-first. “We’re not moving away from our SaaS roots,” affirmed Vembu. We’re simply building a layer of intelligence on top of what we’ve built over the years.” According to him, the systems of record are evolving – first into systems of experience, and now into systems of intelligence.

Zia defines Zoho’s zeal to offer smarter solutions, but it doesn’t make the centrepiece of a new avatar for Zoho. It’s just another layer – just like single sign-on, enterprise search, or low-code extensions, said the tech startup.

The company is also not going to market Zia as ‘next-gen AI’ or ‘sentient copilot’ but rather bundle it with Zoho services as a set of task-specific tools that users can turn to whenever needed at no extra cost, it explained.

That makes Zia LLM different from hyperscalers and VC-backed AI startups often push one-size-fits-all models for computing resources, storage, and networking infrastructure.

And, how is it different from foundational LLMs like GPT-4 and Claude?

These are built to handle anything – from coding to composing sonnets. “We believe that level of ‘emergent behaviour’ isn’t just unnecessary, it’s wasteful for most business use cases,” Vembu told Inc42.

“Smaller, task-specific LLMs aren’t meant to replace general-purpose models across the board because that’s not their goal. What they can do is deliver high performance within narrowly defined domains or use cases,” said Beerud Sheth, CEO of Gupshup. His brainchild enables businesses to build bots and offers a rich messaging service across multiple channels.

LLM Built On Infra, Not On Cloud

The Zia LLM, according to the company, runs on Zoho’s indigenous infrastructure, built over 14 years of investment in proprietary data centres and back-end services. This gives it total control over cost, latency, privacy, and performance that are critical for enterprise clients.

“With the large volume of customers we serve globally, we’ve never faced scalability issues. Customers don’t care whether we’re hosted on this public cloud or that – they care about the performance, functionality, and the value we deliver.”

This self-reliance allows Zoho to maintain flat pricing, without adding AI-based upsell tiers. Unlike its peers that have sent AI features beyond paywalls, Zoho has kept its AI tools bundled within its existing SaaS pricing model.

In India, Zoho’s second-largest market in terms of revenue, the company recorded 32% on-year growth in FY24. Vembu attributes part of this momentum to the AI-led upgrades in the core product suite. “Our AI API usage grew 50% year-over-year to 160 billion calls, and that reflects the customer adoption,” Vembu said.

Flexible Ecosystem, Not Walled Garden

Interoperability becomes crucial as Zoho expands its AI footprint. Zoho has deployed the Model Context Protocol (MCP) for developers and users to mix-and-match agents and applications across platforms.

The internal protocol allows a customer to plug a third-party agent builder into Zoho’s data layers and ensures that its data structure remains accessible and adaptable. “Some customers may have already invested in agent builders or other models. We don’t want to force them to use only Zoho tools,” Ramprakash Ramamoorthy, who is leading Zoho’s big leap into AI, said.

To make the ecosystem more flexible, Zoho has launched the Zia Agent Marketplace. The idea is to enable greater customisability and scalability of its AI ecosystem. This will allow developers and partners to build and publish their own AI agents that can tailor solutions to specific use cases, industries, or geographies.

This open, modular approach decentralises innovation and, instead of relying solely on Zoho to develop new AI agents, third-party contributors can now address niche problems and regional requirements. For instance, early partner interest has emerged around building localised KYC verification tools for different countries, said Vembu.

Zoho continues to invest in Indian language capabilities for speech models. English and Hindi ASR models are live, while more Indic languages are in the pipeline. “English was relatively easy. Hindi took some customisation. But Indic languages are hard because of data scarcity. You also need linguists who know the language intuitively to help us curate datasets. Still, being an Indian company, this Indic focus is critical, especially for SMBs across tier II and III cities,” Ramamoorthy said.

A Thinking Platform, Not A Flashy Chatbot

Zoho’s long-term game is not about building the biggest model or launching the flashiest AI chatbot. It’s about creating deep, verticalised intelligence on its platform. The product roadmap evolves in sync with the rise in mid and large enterprises adopting it. SMB growth remains stable, but acceleration is more in the mid-market.

“The fastest-growing segment for us now is mid and large customers,” the company said. “That’s where the demand for AI-driven digitisation is strongest.”

Zoho, however, doesn’t see AI as a standalone product, rather uses it as an enabler for businesses to digitise every touchpoint. “The future is not about selling one product and leaving the customer to figure it out,” said Vembu. “You have to solve the problem end-to-end. That’s what we’ve always aimed for, and AI is just another tool in that journey.”

Gupshup’s Sheth pointed out the sustainability issue. “Building an in-house LLM stack is a massive investment – from silicon to software – and without a volume play or monetisation strategy, it’s tough to imagine it being cost-competitive in the traditional sense,” he said. “But the calculus changes if it enables better control, regulatory compliance, and faster iteration cycles without relying on third parties.”

[Edited by Kumar Chatterjee]