Introduction
The DPDP bill, 2023 provided for a statutory rights-and-responsibilities-based regime with regard to personal data processing in India. Meanwhile, artificial-intelligence (AI) systems — particularly machine-learning models fuelled by big data — are watching over product, service and process pipelines across industries. At the point where these trends meet, we have growing organisational challenges—which I will describe as a complex compliance environment: AI multiplies the positive and negative impacts of processing personal data on those affected, while DPDP imposes specific responsibilities that determine how personal data can be collected, processed, shared and managed. This note breaks down the DPDP Act’s key mandates for AI powered systems, provides highlights of legal and operational risks faced by corporates and proposed practical solutions — governance, contractual, technical — to mitigate regulatory, operational and reputational risk.
The legal bottom line for A.I.
There are four DPDP Act themes that are of particular relevance for AI systems:
Lawful basis & purpose limitation. Personal data can only be processed under prescribed grounds (i.e., with informed consent, as needed by law/contract, legitimate purposes in terms of the Act). Organizations can’t simply decide to repurpose collected data for one purpose (perhaps, customer service) to train models for some other unrelated purpose completely (like ad targeting, predictive hiring), without establishing a fresh lawful basis or providing notice and new choice to the data principal.
Transparency & notice. Data principals must be informed at the time of collection of its purpose, categories of data collected, treatment period and any potential cross border transfers. For AI, this would entail notices specifically indicating whether an organization is willing to use their data for model training, for profiling or for automated decision-making.
Responsibility & heightened responsibilities of large trustees. The Act envisages creation of “significant data fiduciaries” (SDFs), which carry extra obligations — conducting data protection impact assessments, independent audit, having higher standards of governance and even appointing officers with specialized tasks. AI services with a wide reach, such as major platforms for profiling and monitoring or datasets that cover the entire country/market, would be likely to be among those designated SDFs.
CASES
1. Smt. Sitabai and Anr. v. Ramchandra, (1969) 2 SCC 544
Breach notification & remedies. The Act mandates prompt notice to the Data Protection Board and affected principals of any breaches. For AI, they can include model extraction attacks, training data exposures or accidental exposures of sensitive information through model outputs – which can serve as incidents prompting the Act’s notification obligations and enforcement action.
And these baseline responsibilities directly correspond to core AI lifecycle tasks — data collection, cleaning and labelling; model train and validate; deploy and monitor a model; consume third-party models.
Areas of increased compliance risk with AI
AI systems pose a constellation of risks, some new and some exacerbated from those associated with more classical IT systems:
Secondary use and provenance gaps. A lot of AI models are fed by an aggregated database from multiple sources. If there is no clear provenance (where data came from), then it becomes challenging to assert a legal basis for training or reuse. Past consents may not extend to new AI applications.
Automated decisions and profiling. AI models that “make or automate” decisions about people (for example, hiring, lending to, the price of insurance) impose higher requirements for transparency, contestability and fairness. There is a need for organizations to articulate to regulators, and affected individuals, the logic (14) upon which they have based their automated adverse decisioning results.
Model leakage and re-identification. Model inversion and prompt-extraction are tricks that can expose training data artefacts. Even de-identified datasets can be re-identified by very powerful models, raising the issue that what is said to anonymized data could in fact become personal data in law.
Third-party models and supply-chain risk. By utilizing the (external) LLMs or pre-trained models (as a service), some amount of processing is transferred to the vendors. The fiduciary is still ultimately responsible for what is done on his behalf under the DPDP, and contractual protections as well as audit rights become critical.
Cross-border processing. Model training and inference frequently depends on global cloud infrastructure. And where data (or model parameters that are representations of data) crosses a border, it does so under rules and safeguards; if the rules are not complied with the transfer can be rendered invalid, subject to penalties.
Practical governance and compliance measures
To ensure that AI development and deployment is consistent with DPDP obligations, corporates need a hybrid legal-technical governance programme in place. The measures below are feasible, ranked and operational.
Data & model repository
- Keep a surviving register, which includes:
- Datasets (source, personal categories of data, lawful basis, text of consent or contractual clause),
- Model assets (purpose, inputs/outputs, reporting dates, version numbers, vendors),
- Flows (internal & cross-border), and
- Retention schedules.
This register is the core of control: its power to enable DPIAs, breach response and lawful basis checks.
DPIAs for AI systems
Keep a record of the conducted DPIA for any system processing personal data and all systems taking decisions based on automated methods or profiling. A DPIA should:
- Map harms (privacy, reputational, fairness),
- Record implemented mechanisms (data minimization, pseudonymization, access limitations),
- Define fairness as separate type of error) in the output domain.
- Define procedures of rollback and human review for adverse results.
In the case of SDFs, DPIAs should be more granular and repeated regularly.
Strengthen notices, consent and opt-outs
Revise privacy notices to clearly state AI uses—model training, profiling and automated decision-making—and offer easy, granular choice mechanisms where the law demands. Record and track consent in data provenance logs.
Contractual & procurement controls
Update master services agreements and procurement templates for cloud/ML vendors and data suppliers to contain provisions on:
- DPDP-aligned processing clauses,
- Strict sub-processor obligations, recipients 58 rules and flow-downs,
- Audit and inspection rights,
- Breach notification timelines and responsibilities,
- Requirements about the return or proper disposal of data when a contract ends.
33) Make sure that the vendor SLAs and technical contracts have security obligations (and transparency to a large extent, model cards, lineage, training-data summaries in those scenarios where it’s possible).
Technical controls in ML pipelines
Embed privacy engineering controls:
- Data minimization and schema pruning,
- Differential privacy or enhanced masking where appropriate
- Role-based access control, security both at rest and in motion,
- Locked-down model registries and artifact lineage,
- Logging and explain ability instrumentation (decision logs, model cards and data cards).
Architect ML pipelines such that retraining, data augmentation and third-party model updates cannot run if they bypass automatic policy checks (e.g., signed off lawful basis, consent flags.)
Human oversight & contestability
For high-consequence automated decisions, the introduction of human-in-the-loop review, appeal processes and clear processes for individuals to challenge an outcome. Document how the outputs f models are used by human reviewers and what override they can make.
Risk model-calibrated incident response
Expand the incident response playbook of your organization to encompass model-specific incidents: extraction, prompt leaks, memorized PII in outputs. Create thresholds for statutory notifications and templates regarding regulator and principal communication.
Training, culture & board oversight
Educate product, legal, data science, compliance teams on DPDP basics and AI risk. Make sure the board or a senior risk committee receive regular reporting on inventory of AI, DPIA status, incidents and remediation progress.
Sectoral and cross-border considerations
Regulated industries (finance, healthcare, telecom) will carry additional layers of obligations imposed by sector regulators that mandate higher explainability, audit trails or localisation. For cross-border use-cases, retain legal assessments of destination jurisdictions, consider contractual bridge to moves for transfers and if feasible patternize architectures that can lease sensitive data out of jurisdiction (on-premise or local cloud regions).
Contracts and procurement
Key clauses to discuss with vendors and data providers:
- Clearly communicated instructions and limitation,
- Duty to help with DPIAs and requests from regulators,
- Quick breach reporting and forensic collaboration,
- Sub-process + re-processing train on the model outside of scope of contract prohibition (or controlled approval),
- Indemnification for violations or careless treatment of data,
- Audit and get summaries of training data, as well as the lineage artifacts of a model.
These commercial defences do not substitute for organizational accountability, but are indispensable as risk distributing devices.
Enforcement, litigation and reputational risk
Failing to do so can result in regulatory sanctions, remedial directions and fines, but the greater risk may be loss of trust. AI-driven harms (e.g., biased hiring results; privacy and disclosure) can easily become the focus of news articles and class actions in the jurisdiction affected. A well-articulated and documented governance program is no longer just a defence against regulation; it’s a competitive advantage for reputation.
Conclusion — operationalising compliance in responsible AI
Processing of Personal Data the DPDP Act requires that processing of personal data must be done lawfully, transparently and in a prescribed manner. In addition, AI makes it trickier to comply due to subverting established lines of data separation and providing more sophisticated inferences and extending the envelope on automated decision making. So Indian corporates ought to approach AI governance as an extension of data protection rather than as an afterthought: list your data and models, conduct DPIAs, harden technical pipelines, update contracts to include them and put in place human oversight where relevant. Boards in all companies need to require evidence that AI is being actively monitored and have plans for fixing them. Businesses that are using legal, technical and ethical controls in concert will be much better placed to take advantage of AI’s gains in productivity, while reducing their regulatory, legal and reputational risk.