Sovereign AI Infrastructure Explained for Indian Enterprises: What You Can Control
Sovereign AI is not a model. It is the set of answers to three questions: who can read your data, who can switch your service off, and where the arithmetic runs.
Most enterprise buyers in India arrive with a different belief. They think sovereign AI means picking a cloud region in Mumbai or Hyderabad, or waiting for a national model to replace the foreign one they use today. Both ideas contain some truth, and both skip the part that decides the outcome. A model served from a data centre in Chennai can still be switched off by a company headquartered in another country, read by its operations staff, and fed by a pipeline that quietly sends your documents elsewhere for embedding.
This post takes that belief apart. It sorts the three things people mean by the term, lays out what the Indian government has actually built and what the data-protection rules actually require so far, then gives a five-level scale of control, a short list of dependencies that undermine it, and a configuration that makes a private deployment verifiable instead of merely claimed.
Three Things People Mean by Sovereign AI, and Only One Is Yours to Buy
The same two words are used for three very different projects. Mixing them is the first reason enterprise plans go wrong.
Meaning | Who pursues it | What it looks like | Is it your decision? |
National capability | Government and large research bodies | Public compute, home-grown foundation models, national datasets | No, you can use it but you do not control it |
Legal data residency | Regulators and compliance teams | Personal or sector data stored and processed inside India as the law requires | Partly, the law sets the floor and you choose how far above it to go |
Operational control | Your own engineering and security leadership | You decide who can access the data, which model runs, and when it updates or stops | Yes, entirely, and this is what you can buy or build |
Newspapers and policy documents mostly discuss the first meaning. Lawyers mostly discuss the second. Your architecture decisions sit in the third. A company can satisfy the second meaning without any of the third: data can sit in an Indian region of a global provider, comply with a localisation rule, and still depend entirely on a vendor's terms for access, updates and continuity.
The reverse also happens. A team can run a model on a server under a desk, with complete operational control, and still break the data rules if the pipeline around it logs customer records to an overseas analytics service.
Residency tells you where the data sits. Sovereignty tells you who can act on it. A system can pass the first test and fail the second without anyone noticing until an audit or an outage.
What India Is Building, in Numbers You Can Check
The national programme is real and large, and it is also easy to misread. The IndiaAI Mission was approved by the Union Cabinet in March 2024 with an outlay of about 10,372 crore rupees over five years. It is organised around seven pillars: compute capacity, foundation models, a datasets platform called AIKosh, application development, skills, startup financing, and safe and trusted AI.
The compute pillar is the one that shapes enterprise thinking. The government reported more than 38,000 GPUs onboarded to a shared compute portal by March 2026, and said the shared capacity had grown to more than 45,000 GPUs by June 2026. By August 2026 it reported that 237 projects had used subsidised computing, covering about 93 lakh GPU hours. On the model side, MeitY told Parliament in February 2026 that twelve teams had been shortlisted to build indigenous foundation models.
Two facts about how this works are easy to miss, and both affect you.
First, the Mission does not own the GPUs. Empanelled private cloud companies bought the hardware, and the Mission pays part of the bill when an approved user rents the capacity. An analysis published by ThePrint found that the largest share of the compute subsidy committed up to that point had gone to a small group of foundation-model builders. Second, the programme is aimed at startups, researchers and public-interest projects. An enterprise running an internal document assistant for loan officers is not the intended user and should not plan on being one.
So the national effort helps you indirectly: it grows the supply of domestic GPU capacity, it produces models in Indian languages, and it lowers the cost of renting compute from Indian providers. It does not give your company a sovereign stack. That has to be designed, and the design starts with your own data and your own threat model.
These figures are as publicly reported through October 2026 and they move. Check the IndiaAI and MeitY pages before you quote them in a board paper.
Where the Law Bites: DPDP, Sector Rules and What Has Not Started Yet
The Digital Personal Data Protection Act was passed in 2023, and the Rules under it were notified on 13 November 2025. The government chose a phased start. The first phase took effect immediately and covers the institutional pieces, such as the Data Protection Board. A middle phase runs about twelve months from notification. The operational core, covering notice and consent, the duties of data fiduciaries, the extra duties of Significant Data Fiduciaries and the cross-border provisions, is scheduled for around mid-May 2027, eighteen months after notification. Reports differ by a day on the exact date, so check the notification itself.
Three points in that structure matter for AI infrastructure.
Cross-border transfer is permitted by default under the Act, subject to restrictions the Central Government may notify, and the Rules allow the government to attach conditions that vary by country or recipient. That is not a localisation mandate. It is a power to impose conditions later, which is a different planning problem: you need an architecture that can tighten quickly.
Significant Data Fiduciaries face heavier duties, including an annual impact assessment and audit, a check that algorithmic software does not threaten data principals' rights, and any localisation conditions the government chooses to notify for them. Whether your company will be designated is not something you control.
And sector regulators have their own rules that are already in force and do not wait for the DPDP timeline. The Reserve Bank's direction on storing payment system data in India is the best-known example. Insurance, capital markets and telecom each have their own requirements for particular classes of data. Ask your compliance team which of these apply before you pick an architecture.
Nothing here is legal advice. The practical reading is simple. Between now and mid-2027 you can build for the strictest plausible reading at modest extra cost, or build for today's minimum and rebuild when the rules start. Rebuilding an AI pipeline is more expensive than designing it right the first time.
Five Levels of Control, and What Each One Costs You
Control is a scale, not a switch. Moving up it buys more independence and costs more money, skill and time.
Level | Who can reach your data | Who can switch it off | Exposure to foreign law | Operating burden on you |
1. Public cloud, in-country region | The provider's staff and systems under its terms | The provider | Depends on the provider's home jurisdiction | Low |
2. Dedicated tenancy or confidential computing in-country | The provider, with technical limits on access | The provider | Reduced by technical controls, not removed | Low to medium |
3. Your hardware in a colocation facility | You, plus facility staff for physical access | You, and the facility on contract breach | Limited, depends on who owns the facility | Medium |
4. On-premises in your own building | You | You | Limited to the hardware vendors' support terms | Medium to high |
5. Air-gapped, no route to the internet | Only people physically inside | You | Minimal at run time, updates are manual | High |
A few readings of this table are worth stating.
Level 1 is where most enterprises start, and for much of their data it is sufficient. The question to ask is not whether it is bad but which of your data should never be there.
The jump from level 2 to level 3 is where the character of the project changes. Up to level 2 you are buying a service. From level 3 you are operating hardware, with all that implies: GPU drivers, firmware updates, cooling, spares, and a person who can be called at night.
Level 5 gives the strongest answer to the foreign-law question and the weakest answer to the question of how you patch a vulnerability. Every model update, library fix and driver release has to enter on removable media after inspection. Organisations that need it, such as defence and certain critical-infrastructure operators, accept that cost. Organisations that do not need it should not adopt it for comfort.
A common error is to treat the whole company as one level. Different data deserves different levels. A public-facing marketing summariser can live at level 1 forever. A model that reads KYC documents might belong at level 3. A model touching defence-related designs could belong at level 5. The skill lies in classifying the workloads, then sending each to the cheapest level that satisfies its rules.
Choose the level per workload, not per company. The cost of control is high enough that spending it on data nobody would protect is a form of waste.
The Control Question Hides Four Dependencies
Even at level 4, a deployment can depend on things outside your control. Four dependencies recur.
Dependency | Typical origin | What failure looks like | A mitigation you can afford |
Accelerator hardware | Imported, from a small number of manufacturers | Delivery delays, export-rule changes, scarce spares | Hold spares, avoid single-SKU designs, qualify a second source |
Model weights and licence | Open-weight release or a commercial licence | Licence terms change, a model is withdrawn | Keep a local copy of every version in use, record the licence text |
Software stack | Vendor drivers and libraries, mostly foreign | An update breaks a build, support ends | Pin versions, mirror packages internally, test updates offline first |
Operations and updates | Your staff and your vendor | Key person leaves, vendor stops supporting | Runbooks, a second trained person, a support contract with named response times |
Export-control rules for advanced chips have changed more than once in recent years in the countries that make them, so treat hardware supply as a risk to be managed and not as a settled fact.
The model-weights row deserves attention. Open-weight does not mean unrestricted. Some licences restrict commercial use above a size threshold, restrict certain fields of use, or require acknowledgement in products. Record the licence text next to the model hash. If a licence changes, you want to know which version you were entitled to run.
From these four dependencies comes a test that cuts through slideware: the kill-switch test. Ask what happens to your service if, tomorrow morning, a single named party stops cooperating. For each party in the stack, the cloud provider, the hardware vendor, the model publisher and the software vendor, can you keep running for ninety days? If the answer for any one is no, then the question to answer is how much that matters for this workload, and what it would cost to change the answer.
A Network Policy That Makes Private Verifiable
Claims of privacy are cheap. A deployment is private when you can show that the inference service cannot send anything out and that nothing unexpected is connected to it. In Kubernetes, the starting point is a network policy that denies all egress except name resolution.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: inference-no-egress
namespace: private-inference
spec:
podSelector:
matchLabels:
app: llm-server
policyTypes: [Ingress, Egress]
ingress:
- from:
- namespaceSelector:
matchLabels:
name: internal-apps
ports:
- protocol: TCP
port: 8000
egress:
- to:
- namespaceSelector:
matchLabels:
name: kube-system
ports:
- protocol: UDP
port: 53
The policy lets one internal namespace call the model server on a single port, lets the server resolve names, and blocks everything else. It will only be enforced if the cluster's network plugin supports network policies, which is a point to confirm on day one rather than assume.
A policy covers the model server. It does not cover the pipeline around it. The second piece is a flow inventory: a list of every service in the workflow, where it sends data, and whether that data is personal. A short script turns the inventory into a report that a reviewer can read in a minute.
def residency_report(flows, allowed_countries={"IN"}):
# flows: list of (service, destination_country, carries_personal_data)
bad = [f for f in flows if f[2] and f[1] not in allowed_countries]
return len(flows), len(bad), [b[0] for b in bad]
flows = [("chat-api", "IN", True), ("telemetry", "US", False),
("embeddings-saas", "US", True), ("backup", "IN", True)]
print(residency_report(flows)) # (4, 1, ['embeddings-saas'])
Four flows, one violation: a hosted embeddings service in another country receiving personal data. In a real review the inventory comes from egress logs and service configuration, not from a developer's memory, and the discrepancies between the two are the interesting part.
A sovereignty claim that cannot be demonstrated is a promise. Write the egress policy and the flow inventory first, and let the architecture diagram follow them.
A Representative Composite: A Lender's Document Assistant
This is a representative composite, genericised, built from several enterprise engagements. The numbers are illustrative.
A mid-sized non-bank lender wanted an assistant that answers questions about loan files for forty credit officers: income proofs, KYC documents, property papers. The first prototype used a hosted language model through a public API. It worked well in demos. Then compliance reviewed it and asked three questions nobody had prepared for. Where do the documents go? Who at the model provider can read them? What happens if the provider changes its terms?
The team moved to a private deployment. They chose a mid-sized open-weight model that fit on a single workstation-class accelerator, ran it behind the network policy shown above, and kept retrieval entirely inside the lender's network. It took about eight weeks.
The surprise came in week six. The flow inventory, assembled from egress logs rather than from the design document, showed the retrieval pipeline calling a hosted embeddings service in another country for every uploaded page. The design document did not mention it. A developer had added it in the first week for speed and forgotten it. The documents had been leaving the building the whole time, in a system everyone believed was private.
The fix was a local embedding model, an afternoon of work and a re-index of the corpus over a weekend. The real cost was the confidence lost: the team now treated every claim in the design document as unverified until the egress logs agreed with it.
Quality was the second surprise. The private model was weaker than the hosted one on difficult income-proof reasoning. The team did not close that gap by buying a bigger model. They narrowed the task. The assistant stopped trying to judge creditworthiness and became a fast, cited search over the file, with every answer linking to the page it came from. Officers trusted it more because it claimed less.
The kill-switch test ran at the end of the project. The team switched off outside connectivity for a day. The assistant kept working. The only things that failed were the nightly model-licence check, which phoned home, and a monitoring plug-in that sent anonymous usage statistics. Both were removed.
What Control Costs: Owning Versus Renting, With the Arithmetic
Control has a price, and the price depends on one number more than any other: how many hours a year the hardware does useful work. A server that runs hot all day is cheap per unit of work. A server that sits idle is an expensive ornament.
The model below compares owning one workstation-class inference machine with renting equivalent capacity by the hour. Every input is illustrative. Replace them with your own quotes, your own tariff and your own staff costs. The structure is what to keep: owning has a fixed annual cost that barely moves with use, and renting has a cost that rises with every hour.
def annual_cost_owned(capex, life_years, ops_pct, idle_kw, load_kw, util, tariff, hours=8760):
dep = capex / life_years # straight-line depreciation
ops = capex * ops_pct # support, spares, admin time
avg_kw = idle_kw + (load_kw - idle_kw) * util
energy = avg_kw * hours * tariff
return dep + ops + energy
def breakeven_utilisation(capex, life_years, ops_pct, idle_kw, load_kw, tariff, rent_per_hour, hours=8760):
lo, hi = 0.0, 1.0
for _ in range(60):
mid = (lo + hi) / 2
owned = annual_cost_owned(capex, life_years, ops_pct, idle_kw, load_kw, mid, tariff, hours)
rented = rent_per_hour * hours * mid
lo, hi = (mid, hi) if owned > rented else (lo, mid)
return (lo + hi) / 2
# illustrative inputs: 9 lakh rupees installed, 3-year life, 10 percent a year for support,
# 0.15 kW idle and 0.85 kW under load, 9 rupees per kWh, rented capacity at 200 rupees an hour
print(round(breakeven_utilisation(900000, 3, 0.10, 0.15, 0.85, 9.0, 200.0) * 100, 1)) # 23.7
With those inputs the break-even sits at about 24 percent utilisation. Below it, renting is cheaper. Above it, owning wins, and the gap widens quickly.
Utilisation | Owned machine, per year | Rented capacity, per year | Cheaper option |
10 percent | about 4.07 lakh rupees | about 1.75 lakh rupees | Renting |
24 percent | about 4.15 lakh rupees | about 4.2 lakh rupees | Break-even |
50 percent | about 4.29 lakh rupees | about 8.76 lakh rupees | Owning |
80 percent | about 4.46 lakh rupees | about 14.0 lakh rupees | Owning |
Two honest cautions come with the table. First, utilisation here means hours of paid-equivalent work, and for an internal assistant used in office hours it can sit well below 24 percent unless you schedule batch jobs, document indexing and evaluation runs into the evenings. Second, the comparison is not like for like on control. Renting from a large provider's Indian region gives you level 1 or 2 on the scale above. Owning gives you level 3 or 4. The cost difference is what you pay for control, and the right question is whether the data on this machine deserves it.
If a workload cannot keep the machine busy a quarter of the year, renting is the cheaper road, and the sovereignty argument has to carry the extra cost on its own.
Staff cost is the item most often left out. The 10 percent annual operating allowance in the model stands in for it, and for a first deployment it is optimistic. A machine with a named owner who spends a day a month on patching, monitoring and backups costs real money, and the owner's time is the scarcest input in most Indian engineering teams.
The Questions That Separate Control from Marketing
Vendors use the word sovereign freely. A short set of questions, asked in writing, separates descriptions from commitments.
What you hear | What to ask | Evidence that settles it |
"Your data stays in India" | Where are backups, logs, support access and model telemetry held, and by whom? | A data-flow diagram with every store and the staff roles allowed to reach it |
"We have no access to your content" | Under what technical control is access prevented, and what happens under a legal demand? | Documented key management, who holds the keys, the provider's legal-request policy |
"Fully private deployment" | Does any component call out to an external service for licensing, updates, analytics or embeddings? | Egress logs for a representative week, and the network policy that enforces them |
"Open model, no lock-in" | Which licence applies, which version, and can you keep running if it is withdrawn? | The licence text, a copy of the weights with a hash, a tested restore |
"Indigenous AI" | Which parts are built domestically: model, training data, chips, serving software, operations? | A written bill of materials for the stack, with origin for each layer |
"Always available" | What is the fallback if the vendor stops cooperating, for ninety days? | A runbook, a tested failover, and the contract clause covering termination |
The questions work because they ask for artefacts, not adjectives. A vendor who has built the thing can produce a diagram, a log and a runbook within a week. A vendor who has only described it will send a brochure.
A useful habit is to ask the same questions of your own team. Internal projects drift from their design documents as fast as vendor deployments do, and the composite above showed an embeddings call nobody remembered adding.
An Evidence Pack for Your Auditor
If a regulator, a customer or your own board asks you to show that a deployment is private and under your control, you want to hand over a small, current folder. Six artefacts cover most requests.
The data-flow inventory comes first. It lists every service in the AI workflow, what it receives, where it sends it, and whether personal or regulated data is involved. Build it from egress logs and configuration, then reconcile it with the design document, and record the differences.
The enforcement record is the second. This is the network policy, evidence that the cluster enforces it, and the output of a test that tries to reach an outside address from the inference namespace and fails.
Third is the model and licence register: each model in use, its version, its hash, its licence text, the date you downloaded it and where the verified copy is stored.
Fourth is the kill-switch test record. A dated note of the day you cut outside connectivity and what broke, plus the fixes. Repeat it twice a year, and after any major change.
Fifth is the patch and update log: what was updated, when, from which source, who approved it and how it was tested. For air-gapped systems this includes the inspection record for each piece of media.
Sixth is the access review: who has administrative access to the machines, the model store and the logs, reviewed quarterly, with removals recorded.
None of these are exotic, and a team that keeps them current spends perhaps an hour a week on them. The payoff is that an audit, a customer questionnaire or an incident review turns into a folder, and not a month of reconstruction.
Evidence beats assurance. Whatever you would want to see from a vendor, build for yourself and keep up to date.
Indian-Language Models and What They Change in Your Sizing
One aim of the national programme is models that handle Indian languages well, and for many enterprises that is the practical meaning of sovereign AI: customers and staff who write in Hindi, Telugu, Tamil or Marathi and expect the assistant to follow. Two engineering consequences belong in your plan from the start.
The first is tokens. A model splits text into tokens before it reads it, and tokenisers trained mostly on English break Indic scripts into many more pieces per word. The ratio depends on the tokeniser and the language, and it can be several times higher than for English text. More tokens per sentence means more cache memory per conversation, more computation per answer and slower perceived speed, so any sizing table, including the ones in our companion hardware guide, needs a multiplier for non-English use. Measure it rather than assuming: take a hundred real sentences in your users' language, run them through the candidate model's tokeniser, and compare the count with the English equivalents.
The second is quality. Evaluate in the language your users write, on your own documents. A model that scores well on a public benchmark can still stumble on the legal register of a loan agreement or the shorthand in a field-staff message. Domestic models may be stronger in some languages and weaker on multi-step reasoning, and the only way to know is a test set built from your own material.
Plan for both. Keep the serving layer model-agnostic, with the model behind a stable interface, so that a domestic model can replace the incumbent the day it beats it on your evaluation set. That one design choice protects you against licence changes, supplier problems and the steady arrival of better models, and it costs almost nothing if made on day one.
When Sovereign Is the Wrong Requirement
Not every AI workload needs this, and treating all of them as if they do wastes the money you need for the ones that do.
If the data is public or already published, a hosted model is simpler and usually better. Summarising a public tender document does not need an air gap.
If you need the strongest available model for a task and the data is not sensitive, a hosted frontier model may be the right answer, and no private open-weight model will match it today.
If the workload is bursty, a private cluster sits idle most of the month while you pay for the hardware. Renting capacity when needed is cheaper. Utilisation arithmetic should come before the purchase order.
If you have no one to operate it, a private stack degrades. Drivers drift out of date, nobody checks the logs, and a hardware fault waits for a vendor that is not under contract. Level 3 and above need a named owner.
And if the regulation you are worried about has not started, spending to the strictest reading may still be right, but it should be a decision with a price attached, not an assumption.
A Ninety-Day Path
Inventory the flows. List every AI-related service and what it sends where. Use egress logs, not memory. Mark personal and sector-regulated data.
Classify the workloads. For each, decide the control level it needs, using your compliance team's reading of the rules in force and the ones coming.
Run the kill-switch test on paper. For each workload, name the four dependencies and the answer to the ninety-day question.
Pilot one workload at the level it needs. Choose one with real value and moderate risk, so the pilot teaches you without endangering anything critical.
Make it verifiable. Write the egress policy, enforce it, and add the flow report to your release checklist.
Plan operations. Name an owner, a second person, a patching cadence, a spares plan, and a rollback path for models.
Review in six months against the DPDP timeline and any sector notifications.
Who Can Switch It Off?
The three questions from the start have not changed. Who can read your data. Who can switch your service off. Where the arithmetic runs.
A national programme can lower the price of compute. A cloud region can satisfy a localisation rule. A good model can answer a hard question. None of them can answer those three questions for you, because the answers depend on how your own system is built and on what you are prepared to operate.
Draw the flows. Test the switch. Then decide how much control each piece of data deserves.
Related service: On-Premise and Edge AI Solutions
EurthTech is an engineering-first product design and embedded systems company with a development center in Hyderabad. To discuss a project, write to connect@eurthtech.com or see https://www.eurthtech.com/case-studies.
Comments