By Vikrant Rana, Anuradha Gandhi and Prateek Chandgothia
Introduction
The Final Digital Personal Data Protection Rules (hereinafter referred to as ‘DPDP Rules/ The Rules’)[1] notified by the Ministry of Electronics and Information Technology (hereinafter referred to as ‘MeitY’) operationalized the Digital Personal Data Protection Act, 2025 (hereinafter referred to as ‘DPDPA/ The Act’) fills a major legal vacuum regulating the governance of digital personal data and data privacy in India. However, a major question still looms large as the application of The Act on Artificial Intelligence (hereinafter referred to as ‘AI’) systems remains a legal-technical grey area.
Applicability of DPDPA on AI systems
The IndiaAI Governance Guidelines[2] (hereinafter referred to as ‘guidelines) released by MeitY in early November addressed the concerns around how AI will be regulated in India. The crux of the guidelines indicated that the Government intends to extend the application of existing law rather than enacting a new law to regulate AI systems. Additionally, the report also states that AI companies would be required to comply with sectoral regulations as notified from time to time by respective regulators.
The Guidelines discussed the application of the DPDPA on AI systems and stated that the use of personal data without user consent to train AI models is governed by the DPDP. In the context of AI systems, the Guidelines says that obligations of consent, purpose limitation, and data minimization would have direct bearing on AI model training and deployment. DPDPA prohibits processing of personal data without consent, requires safeguards against misuse of sensitive data, and empowers the Data Protection Board to investigate harms caused by misuse of AI-driven profiling. These provisions create accountability pathways for AI developers and deployers handling personal data at scale.
Reading this Report in Conjunction with Section 5 of the DPDPA and Rule 3 of the Rules, it is conclusive, that specific consent must be obtained for collecting and personal information for the purpose of training of AI models. Additionally, AI systems must implement reasonable security safeguards as mentioned under Rule 6 of the DPDP Rules. These safeguards are applicable across sectors.
(To read more on the Guidelines, refer to – https://ssrana.in/articles/meity-unveils-indias-approach-towards-regulating-artificial-intelligence/ )
Algorithmic Softwares must not risk the Rights granted under DPDPA
Rule 13 of the DPDP Rules prescribes the additional obligations of the Significant Data Fiduciaries (hereinafter referred to as ‘SDF’). In conjunction with the other obligations, an SDF must observe due diligence to verify that technical measures including algorithmic software adopted by it are not likely to pose a risk to the rights of Data Principals. This includes AI systems used for hosting, display, uploading, modification, publishing, and transmission, storage, updating or sharing of personal data processed by it. This means that the processing of personal information using an AI system must not adversely affect the rights of the Data Principals under the DPDPA.
Rights of Data Principals under the DPDPA
Section 11 to 14 of the DPDPA read with Rule 14 of the DPDP Rules grant various rights to the Data Principals which includes:
- Right to access information about personal data
- Right to Correction and erasure of personal data
- Right to grievance redressal
- Right to nominate
While the right to grievance redressal, access and nomination may be fairly simpler to implement when compared to the Right to correction and erasure. This is majorly due to the black box nature of the AI systems
(To read more on the DPDPA and the Rules, refer to – https://ssrana.in/articles/meity-notifies-final-digital-personal-data-protection-rules-2025/ )
Is Right to Erasure feasible in AI systems?
Unlike conventional data repositories such as spreadsheets or structured databases, large language models (LLMs) do not preserve information in discrete, accessible rows or columns. Instead, they operate by adjusting probabilistic weights across billions of parameters, a process informed by the training data to which they have been exposed. Once incorporated, that data is not stored in a manner that permits straightforward retrieval, indexing, or deletion.
To illustrate, if an individual’s name or personal information were included in the corpus used to train a foundational model, the resulting influence of that data is diffused throughout the model’s architecture. It manifests as statistical impressions rather than identifiable records. Consequently, there exists no practical mechanism to surgically excise that individual’s contribution without undertaking a complete retraining of the model. In legal terms, the data is not “retained” in the conventional sense but is instead embedded in a way that resists isolation or removal.[3]
(To read more on data processing by AI systems, refer to – https://ssrana.in/articles/issue-of-prompt-injections-open-ais-new-atlas-browser-faces-critical-cybersecurity-threats/)
This raises some key questions:
- Does this mean that the personal information once embedded in the AI system cannot be identified and extracted? Let’s discuss.
- If the personal information is not stored as identifiable records, then why is there a need to delete it?
On December 17, 2024, the European Data Protection Board (hereinafter referred to as ‘EDPB’) released the Opinion 28/2024 (hereinafter referred to as ‘Opinion’) on certain data protection aspects related to the processing of personal data in the context of AI models.[4] The EDPB in this Opinion answered a key question – when and how an AI model can be considered as ‘anonymous?
The EDPB clarified that when personal data is used to train an AI model, it is absorbed into the AI model’s parameters along with such training data, which are expressed as mathematical patterns. These patterns are not identical to the original data points, but they may still carry pieces of the original information.
That means, in some cases, information about real people whose personal data was used in training could be pulled out of the model, either directly or indirectly. If such information can be obtained using methods that are reasonably likely to be used, then the model cannot be considered anonymous.
(To read more on concerns around data privacy in AI systems, refer to – https://ssrana.in/articles/openais-report-triggers-ethical-ai-concerns-1-2-million-users-seek-self-harm-related-advice-from-chatgpt/)
Then how can your PI be erased from an AI system?
There is no clear or specific answer to this question. Some of the methods that may be followed are[5] –
- Differential Privacy – This method protects people’s data by adding random “noise” to the training dataset. Researchers can still get useful statistics, but it becomes almost impossible to figure out any one person’s information from the results. The trade‑off is that models using this technique are usually less accurate because of the added noise.
- Algorithmic Destruction/ Machine Unlearning – This approach tries to remove specific data points from a trained model without starting over. The challenge is that there’s no agreed‑upon way to measure how effective it is, and for large, complex models, making sure all traces of the data are gone is extremely difficult.
Originally, this right was designed for an internet where data was stored in files or databases. Modern AI systems don’t just store data, they transform and regenerate it. That makes fully applying the right to be forgotten in AI hard, if not impossible. Still, combining techniques like differential privacy and machine unlearning could give people more control over their personal data in the age of generative AI.[6]
The Caveat
Under the DPDPA, the right to erasure gives individuals the right to have their personal data removed when it is no longer necessary or consent is withdrawn, but applying this right to AI systems raises a tension with the Constitutional Right to Trade and the government’s pro‑innovation policy stance. Techniques such as algorithmic destruction or machine unlearning are costly, time‑consuming, and not fully reliable, while differential privacy can reduce accuracy and thereby affect competitiveness. In practice, regulators and courts are likely to apply a proportionality and reasonableness test:
- If erasure is technically feasible without imposing disproportionate burdens, the individual’s right will prevail; or
- If compliance would cripple innovation or trade, alternative safeguards may be accepted.
This is also demonstrable from the fact that:
- The DPDP Rules do not provide for a specific scope of ‘due diligence’ required for ‘algorithmic softwares’ , and
- The obligation of algorithmic software due diligence is limited to SDFs under the DPDPA and not all data fiduciaries.
Thus, while the right to erasure under the DPDPA can override commercial interests, its enforcement will be tempered by practical limits and the government’s innovation‑friendly approach.
[1] https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc20251117695301.pdf
[2] https://static.pib.gov.in/WriteReadData/specificdocs/documents/2025/nov/doc2025115685601.pdf
[3] https://hai.stanford.edu/news/privacy-ai-era-how-do-we-protect-our-personal-information
[4] https://www.edpb.europa.eu/system/files/2024-12/edpb_opinion_202428_ai-models_en.pdf
[5] https://www.sciencedirect.com/science/article/pii/S026736492300095X
[6] https://www.techpolicy.press/the-right-to-be-forgotten-is-dead-data-lives-forever-in-ai/