By Tasmita Sengupta & Sohom Banerjee
SEBI’s proposed AIsafeguards will mark the beginning of a new era in financial regulation, one concerned not only with institutions and transactions, but also how much freedom machines should have. Financial markets have always rewarded speed. The challenge now is that speed is being combined with intelligence and autonomy, often with limited human involvement. On August 19, SEBI Chairman Tuhin Kanta Pandey indicated that the regulator will issue guidelines on the responsible use of artificial intelligence (AI) and machine learning (ML) in the securities markets. The proposed framework is expected to include human oversight, stronger data controls and a “kill switch” that can stop an automated system when needed.
As a frontrunner in financial market reforms, SEBI deserves credit for recognising AI as a market reality requiring proportionate regulation. Already used in trading, surveillance, risk management and compliance, these systems can amplify technical failures at great speed. The regulatory question therefore encompasses how much decision-making authority AI should receive, what safeguards must govern it and when humans should intervene. Historically, regulators have supervised the market and now shall also supervise automated decision-making. A sophisticated system may analyse information, recommend a course of action, execute trades and adjust its behaviour while interacting with other automated systems.
When this happens in milliseconds, traditional human supervision mechanisms become obsolete.This might be recognised as a new version of the principal-agent problem. A financial institution delegates a task to an algorithm because it can act faster and process more information than a human as market tends to move faster. But the algorithm neither bears legal responsibility nor suffers reputational damage when it fails. SEBI’s June 2025 consultation paper recognised several parts of this problem. It proposed continuous monitoring, audit trails, model versioning, independent assessment, fallback arrangements and seniormanagement accountability. An automated model may behave differently after deployment when market conditions, datasets or its operating environment changes.
Therefore, a onetime approval cannot guarantee its continued safety. The risk extends beyond the failure of an individual algorithm. The Financial Stability Board has warned that widespread reliance on the same datasets, cloud services, foundation models or technology vendors could increase market correlations and operational vulnerabilities. The danger may not be one rogue machine. It could be hundreds of functioning systems reacting similarly to the same shock.SEBI can strengthen its proposed framework in six ways. First, regulation should reflect autonomy and materiality, rather than treating every AI application alike. An internal chatbot should not face the same requirements as a system that independently executes large trades. SEBI could create an “AI Autonomy Ladder” or graded risk categories, ranging from tools that only provide information, to those that make recommendations, execute decisions within fixed limits, or operate with substantial independence.
Testing, disclosure and oversight requirements should become stricter as the system’s authority and market impact increases. Second, a kill switch must be treated as a protocol, not simply as a button. Firms should identify who is authorised to activate it, what warning signs would trigger intervention, how quickly action must be taken and what happens to pending orders and open positions. These procedures should be tested through periodic drills. A safeguard that has never been tested may fail precisely when it is needed most.
Third, SEBI could maintain a confidential Systemic AI Dependency Register. Market institutions should disclose the external models, cloud providers, datasets and vendors supporting their most important functions. The objective would not be to expose proprietary strategies but to allow the regulator to see something individual firms cannot, whether apparently independent market participants are actually dependent on the same technological infrastructure. SEBI’s consultation already identifies concentration among a limited number of AI providers as a potential systemic risk. Fourth, India should consider market-wide stress exercises involving exchanges, clearing corporations, brokers and major institutional investors.
Possible scenarios could include a corrupted data feed, an incorrect trading signal, a cloud outage, a liquidity shock or the simultaneous withdrawal of several automated models. The purpose should be to assess not only whether individual institutions remain stable, but also whether their combined responses could unsettle the wider market. Such risks were even evident before the emergence of modern AI. In 2012, a software deployment failure at Knight Capital sent more than four million erroneous orders into the US market in 45 minutes and caused losses exceeding $460 million, according to the US Securities and Exchange Commission. The episode showed how quickly automated execution can outrun human correction. An AI system that adapts to incoming data could make the chain of failure still harder to reconstruct, which is why market-wide drills and reliable audit trails matter.
Fifth, meaningful human oversight should be judged by the ability to intervene, not merely by whether a person is present. Requiring human approval for every machine-generated decision would be unrealistic at market speed. High-risk systems instead need exposure limits, automatic alerts, clear escalation thresholds and trained officials who can intervene within a specified time. Finally, SEBI should evaluate the regulations after their implementation. Such an assessment should examine incidents and near misses, investor complaints, false alerts, compliance costs, effects on smaller intermediaries and whether the rules are unintentionally favouring large technology providers or discouraging useful innovation. Regulation governing learning systems must itself be capable of learning from experience.
SEBI’s broader thinking on cyber resilience reinforces this direction. Chairman Pandey has recently argued that cybersecurity can no longer be treated merely as an IT issue; it is a board-level, business-continuity and market-integrity concern. AI deserves the same elevation.The objective is not to slow the adoption of new technology. Innovation in financial markets becomes sustainable when confidence in its safeguards grows alongside its capabilities.The lasting importance of SEBI’s initiative may therefore lie beyond the colourful phrase “kill switch”. It could establish a broader principle: no machine operating in a systemically important market should exercise authority that people cannot monitor, reconstruct, limit and, when necessary, withdraw. That is not anti-innovation. It is the governance that responsible innovation now requires.
*Tasmita Sengupta is a senior research associate at CUTS International, Centre for Competition, Investment and Economic Regulation (CUTS CCIER).
*Sohom Banerjee is a tech professional with over 8+ years of experience across public policy, technology, infrastructure, trade, media, telecommunication and economic development.
This article can also be viewed at:
https://sundayguardianlive.com/