How to Secure LLMs Against AI Cyber Risks | EY - India

How to Secure LLMs Against AI Cyber Risks | EY - India EY Logo Insights Latest AI insights: GenAI, agentic AI and AI trends People and workforce: Services and latest insights Latest insights: Risk services, compliance and trends Latest insights: Sustainability and ESG trends Technology consulting services and solutions Transformation Realized series: CEO-led business transformation Services Tax services and solutions Global trade insights: Tax perspectives Tax compliance services and solutions Tax planning services and solutions - Direct and indirect tax India tax platform: Digital tax solutions and real-time insights Digicorporatetax for corporate tax compliance GST compliance with EY DigiGST EY GTA: Global trade compliance and supply chain platform AI Tax Hub: Tax transformation AI solution Consulting services and solutions AI consulting services and solutions Cybersecurity consulting services and solutions Technology consulting services and solutions Risk Consulting Services and Solutions People and workforce: Services and latest insights AI solutions AI academy: Latest AI training programs Asterisk: AI led supply chain management tool LLM model for BFSI sector: LLAMA 3.1-8B EY Competency Connect: AI-led talent assessment platform Customer Service & Sales Acceleration Suite: AI-led customer excellence Strategy and Transactions Corporate finance advisory services Investment banking advisory services Debt and Special Situations Services Private equity advisory services Valuation, Modeling and Economics Services M&A Due Diligence Consulting Services Strategy by EY-Parthenon Corporate and growth strategy consulting services Commercial strategy consulting services Digital strategy consulting services Industry strategy consulting services Strategy consulting services Transaction strategy and execution services Technology consulting services and solutions Alliances: EY partner ecosystem Blockchain consulting services EY.ai - AI-powered innovation and transformation Tax technology and data services Technology transformation consulting services Risk Consulting Services and Solutions Financial risk management services Digital and Technology Risk Management Enterprise Risk Sustainability services and solutions Climate change and sustainability services Climate & Decarbonization Services Environment, health and safety (EHS) services Supply chain transformation services Global renewables industry: Insights and services Sustainability tax consulting services Sustainable Finance Services | Green Finance ESG compass: Advanced ESG reporting, solutions and tools People and workforce: Services and latest insights Change management and experience services HR transformation services Integrated workforce mobility Learning and development consulting services Recognition and reward advisory AI academy: Latest AI training programs EY Competency Connect: AI-led talent assessment platform Managed services and solutions Managed services: Cybersecurity Managed services: Financial accounting and corporate reporting Risk Managed Services for Businesses Managed services: Sustainability Managed services: Tax Assurance services and solutions Climate change and sustainability services Financial accounting and advisory services Forensic and integrity - Services and solutions Industries Consumer products and retail: Latest industry insights Future consumer index: Latest consumer behavior insights Energy and resources: Latest industry insights Energy and resources strategy consulting services Mining and metals consulting services and insights Oil and gas consulting services and insights Power and utilities consulting services and insights Financial services: Latest industry insights Banking and capital markets consulting services Insurance consulting services Payment Services and Solutions Wealth and asset management consulting services Government and public sector: Latest insights Digital Government & Public Sector Services Education consulting services Human and social services Infrastructure consulting services Innovative healthcare consulting services and solutions Life Sciences industry solutions and consulting services Private equity - Investment, strategy and operations Digital consulting services in private equity Exit readiness and IPO preparation PE deal origination and opportunity sourcing Private equity value creation services Technology, media and entertainment and telecommunications Media and Entertainment industry: Latest insights Technology sector: Latest insights Telecommunications sector: Insights, surveys and Services Careers About EY-Parthenon careers Experienced job search Early careers opportunities How to join us - Careers and opportunities How we hire - EY recruitment process and tips Interview tips - Succeed in EY hiring process What we look for - EY hiring criteria and values Job search Contract opportunities Job search Students and entry-level Talent community What it's like to work here Diversity, equity and inclusiveness Flexibility and mobility Personalized career development People stories - Employee insights and experiences What you can do here Careers in Assurance Careers in core business services Careers in consulting Careers in financial services Careers in global delivery services Careers in people advisory services Careers in tax - Advisory and compliance jobs Technology jobs and careers at EY About us Alliances: EY partner ecosystem Alumni Connect with us Locations Newsroom Our purpose Our people Our values Insights\r\n"}}" id="rich-text-71fd1871910b" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Insights Read More Read Less Asking the better questions that unlock new answers to the working world's most complex issues.

\r\n"}}" id="rich-text-6e805f5c96cd" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Asking the better questions that unlock new answers to the working world's most complex issues.

Read More Read Less Explore See more

\r\n"}}" id="rich-text-ad573cc50c57" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See more

Read More Read Less Highlights\r\n"}}" id="rich-text-b4d34d9f753b" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Highlights Read More Read Less

Tax

Expectations from Union Budget 2026: Agri sector’s access to finance

20 Jan 2026Amit Bajaj

Tax

Expectations from Union Budget 2026: Ensuring food security

16 Jan 2026Amit Bajaj

 

\r\nEY podcasts

\r\nEY webcasts

\r\nCase studies

\r\nTechnology solutions

\r\n"}}" id="rich-text-1fa5efbab794" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text">  

EY podcasts

EY webcasts

Case studies

Technology solutions

Read More Read Less Services\r\n"}}" id="rich-text-b54b8a52506e" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Services Read More Read Less EY helps clients create long-term value for all stakeholders. Enabled by data and technology, our services and solutions provide trust through assurance and help clients transform, grow and operate.

\r\n"}}" id="rich-text-ba82bdbdb9ab" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> EY helps clients create long-term value for all stakeholders. Enabled by data and technology, our services and solutions provide trust through assurance and help clients transform, grow and operate.

Read More Read Less Explore See more

\r\n"}}" id="rich-text-4e610e760d87" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See more

Read More Read Less Spotlight\r\n"}}" id="rich-text-c9015452f163" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Spotlight Read More Read Less EY.ai - A unifying platform

\r\nTechnology transformation

\r\nEY wavespace™

\r\n"}}" id="rich-text-c98e9d970858" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> EY.ai - A unifying platform

Technology transformation

EY wavespace™

Read More Read Less Industries\r\n"}}" id="rich-text-a80a477f99d3" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Industries Read More Read Less Discover how EY insights and services are helping to reframe the future of your industry.
\r\n

\r\n"}}" id="rich-text-cf89c34d6383" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Discover how EY insights and services are helping to reframe the future of your industry.

Read More Read Less Explore See more

\r\n"}}" id="rich-text-1c524e126f38" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See more

Read More Read Less Case studies\r\n"}}" id="rich-text-747e08e9bef8" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Case studies Read More Read Less

Tata Steel

How Tata Steel became a leader in risk management in the steel sector

18 Dec 2025EY India

HUL’s Suvidha Centres

How HUL’s Suvidha Centres are transforming sustainable sanitation in Mumbai

27 Aug 2025EY India

Telecom

How a global telecom company elevated its SAM maturity with EY Atom

05 Mar 2025EY India

See all

\r\n"}}" id="rich-text-80dd217d3871" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See all

Read More Read Less Careers\r\n"}}" id="rich-text-497afaba73b9" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Careers Read More Read Less We bring together extraordinary people, like you, to build a better working world.

\r\n"}}" id="rich-text-7586edc04bdc" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> We bring together extraordinary people, like you, to build a better working world.

Read More Read Less Explore See more

\r\n"}}" id="rich-text-cc262bdbc063" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See more

Read More Read Less Spotlight\r\n"}}" id="rich-text-79679a1de880" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Spotlight Read More Read Less Experienced professionals

\r\nStudent and entry level programs

\r\nTalent community
\r\nCareers in EY Global Delivery Services

\r\n"}}" id="rich-text-2d86f095fbae" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Experienced professionals

Student and entry level programs

Talent community
Careers in EY Global Delivery Services

Read More Read Less About us\r\n"}}" id="rich-text-53d314f72001" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> About us Read More Read Less At EY, our purpose is building a better working world. The insights and services we provide help to create long-term value for clients, people and society, and to build trust in the capital markets.

\r\n"}}" id="rich-text-e9975a56f1cb" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> At EY, our purpose is building a better working world. The insights and services we provide help to create long-term value for clients, people and society, and to build trust in the capital markets.

Read More Read Less Explore See more

\r\n"}}" id="rich-text-f889db26554e" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> See more

Read More Read Less Top news\r\n"}}" id="rich-text-8f541d40235a" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Top news Read More Read Less

Press release

Deepinder Goyal awarded the EY Entrepreneur Of The Year™ 2025

25 Feb 2026EY India

Press release

India Inc. projects 9.1% salary increase in 2026 as compensation becomes sharper, more skills-led: EY Future of Pay 2026 Report

23 Feb 2026EY India

Press release

Private credit market records investments of US$3.4 Billion in H2 2025: EY Private Credit Report

18 Feb 2026EY India

Search search

close search See all results for ' '

No results have been found

Topics

See All

General

See All

People

See All

Recent Searches

Trending

FY27 Budget: Downward expenditure adjustments and slowing fiscal consolidation FY26 tax reforms cut revenues by INR1.925L cr; FY27 sees slower fiscal consolidation, lower state transfers, and subdued nominal GDP growth. 25 Feb 2026 Tax Redefining operational due diligence in India’s evolving deal landscape Operational Due Diligence in India is evolving from risk mitigation to execution certainty, shaping value creation in private equity and M and A deals. 25 Feb 2026 Strategy and Transactions How AI is becoming central to oil and gas finance strategy AI is helping oil and gas CFOs unlock trapped working capital, improve cash flow resilience and fund energy transition through predictive and agentic automation. 24 Feb 2026 My EY My EY Select your location

Local sites How companies can secure language models against emerging AI cyber risks Mubin Shaikh

Partner, Cybersecurity Consulting, EY India

7 minute read 07 Apr 2025 Related topics AI Cybersecurity Facebook Twitter LinkedIn "> Link Copied As AI adoption grows, so do cyber threats. Organizations must secure large language models (LLMs) against data leaks and prompt injections. \r\n"}}" id="rich-text-3664250c3a0f" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> As AI adoption grows, so do cyber threats. Organizations must secure large language models (LLMs) against data leaks and prompt injections.  Read More Read Less In brief

\r\n\r\nThe global AI market is projected to reach US$1,811.75 billion by 2030.\r\nBy 2025, an estimated 0.75 billion applications will be powered by LLM technology.\r\nOrganizations must implement proactive security measures to mitigate LLM-related information security risks.\r\nTo harness AI’s potential responsibly, organizations must protect against threats and build a secure, ethical AI ecosystem.\r\n\r\n"}}" id="rich-text-2302c6236b06" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> In brief

The global AI market is projected to reach US$1,811.75 billion by 2030.By 2025, an estimated 0.75 billion applications will be powered by LLM technology.Organizations must implement proactive security measures to mitigate LLM-related information security risks.To harness AI’s potential responsibly, organizations must protect against threats and build a secure, ethical AI ecosystem. Read More Read Less With the rapid evolution of technology, Large Language Models (LLMs) have become integral to our lives, transforming industries and daily interactions. The global LLM market is projected to grow from $1.59 billion in 2023 to $259.817 billion in 2030.

\r\n"}}" id="rich-text-84e9cf43795d" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> With the rapid evolution of technology, Large Language Models (LLMs) have become integral to our lives, transforming industries and daily interactions. The global LLM market is projected to grow from $1.59 billion in 2023 to $259.817 billion in 2030.

Read More Read Less From ChatGPT-inspired creativity to sophisticated business chatbots, AI solutions are reshaping work, communication and problem-solving. As dependency on AI grows, so does automation. With AI's expanding role, AI-driven cybersecurity and compliance standards become crucial to ensuring responsible adoption. This silent transformation underscores the essential nature of AI and LLMs in business, emphasizing the need for secure and ethical implementation.

\r\nOrganizations across industries are integrating AI into their core business operations to streamline workflows, automate tasks and improve decision-making. By 2026, over 80% of enterprises will have integrated AI automation and GenAI-enabled applications into their core functions. According to the AI in Action 2024 report, 67% of surveyed leaders reported a 25% revenue increase due to AI integration. However, with AI's expanding role, significant security risks must be addressed to fully harness its potential without compromising safety.

\r\n"}}" id="rich-text-5331fb4a3cbf" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> From ChatGPT-inspired creativity to sophisticated business chatbots, AI solutions are reshaping work, communication and problem-solving. As dependency on AI grows, so does automation. With AI's expanding role, AI-driven cybersecurity and compliance standards become crucial to ensuring responsible adoption. This silent transformation underscores the essential nature of AI and LLMs in business, emphasizing the need for secure and ethical implementation.

Organizations across industries are integrating AI into their core business operations to streamline workflows, automate tasks and improve decision-making. By 2026, over 80% of enterprises will have integrated AI automation and GenAI-enabled applications into their core functions. According to the AI in Action 2024 report, 67% of surveyed leaders reported a 25% revenue increase due to AI integration. However, with AI's expanding role, significant security risks must be addressed to fully harness its potential without compromising safety.

Read More Read Less The dilemma of trust — why AI’s strength is also its weakness?\r\nBy 2025, an estimated 0.75 billion applications will be created using LLMs, automating 50% of the digital processes. This shift marks a new era for LLM security, emphasizing the need to safeguard systems, applications and business against malicious actors.

\r\nWith the growth of AI, there is a higher risk of model vulnerabilities being exploited by malicious actors. As AI adoption increases, so do AI-powered cyber threats. The wider the use of AI, the larger the attack surface, making security a must-have. The researchers at an AI cybersecurity firm noted a 135% rise in LLM-powered phishing attacks in 2023, showcasing various cases of LLM misuse.

\r\nRecently, security researchers discovered that GenAI chatbots could be manipulated through indirect prompt injection attacks, potentially allowing third-party attackers to distribute malicious documents and emails to target accounts and compromising the integrity of the responses.

\r\nData leakage in LLMs is a significant concern, due to various factors. From simple prompt injections to data poisoning, LLMs can be exploited in multiple ways. They may leak exact snippets of their training data (training data regurgitation), reveal information through clever prompts (prompt hijacking), or be manipulated through carefully designed attacks such as model-based parameter manipulation. The challenge is not just building smart AI but securing it against ever-evolving threats.

\r\nNations have started developing AI regulations – most notably the European Union’s AI Act (2024), which applies to both EU member states and non-EU entities offering AI systems within the union. Despite these efforts, the AI regulatory landscape largely remains insufficient, creating challenges in intellectual property, accountability and ethical AI considerations. Critical issues such as the ownership of AI-generated content, liability for misinformation and potential societal impacts remain unaddressed. 

\r\nIn India, the AI regulatory landscape lacks specific codified laws, resulting in LLM compliance challenges. However, two pivotal frameworks are guiding technological development: The National Strategy for Artificial Intelligence and the Principles for Responsible AI. These frameworks represent initial steps toward developing a structured approach to AI governance in enterprises.

\r\nWhile regulatory frameworks lay the groundwork for responsible AI development, ensuring AI security remains a challenge. Beyond policies and technical safeguards, the true test of AI security lies in the human oversight.

\r\nFor all its complexity, encryption and obfuscation, the weakest link in AI security is not AI itself but the humans behind it. AI is perceived as intelligent, leading to overreliance and trust. However, AI does not "understand" security—it merely follows rules, which can be manipulated.    

\r\n"}}" id="rich-text-3b0259a5ac3b" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> The dilemma of trust — why AI’s strength is also its weakness? By 2025, an estimated 0.75 billion applications will be created using LLMs, automating 50% of the digital processes. This shift marks a new era for LLM security, emphasizing the need to safeguard systems, applications and business against malicious actors.

With the growth of AI, there is a higher risk of model vulnerabilities being exploited by malicious actors. As AI adoption increases, so do AI-powered cyber threats. The wider the use of AI, the larger the attack surface, making security a must-have. The researchers at an AI cybersecurity firm noted a 135% rise in LLM-powered phishing attacks in 2023, showcasing various cases of LLM misuse.

Recently, security researchers discovered that GenAI chatbots could be manipulated through indirect prompt injection attacks, potentially allowing third-party attackers to distribute malicious documents and emails to target accounts and compromising the integrity of the responses.

Data leakage in LLMs is a significant concern, due to various factors. From simple prompt injections to data poisoning, LLMs can be exploited in multiple ways. They may leak exact snippets of their training data (training data regurgitation), reveal information through clever prompts (prompt hijacking), or be manipulated through carefully designed attacks such as model-based parameter manipulation. The challenge is not just building smart AI but securing it against ever-evolving threats.

Nations have started developing AI regulations – most notably the European Union’s AI Act (2024), which applies to both EU member states and non-EU entities offering AI systems within the union. Despite these efforts, the AI regulatory landscape largely remains insufficient, creating challenges in intellectual property, accountability and ethical AI considerations. Critical issues such as the ownership of AI-generated content, liability for misinformation and potential societal impacts remain unaddressed. 

In India, the AI regulatory landscape lacks specific codified laws, resulting in LLM compliance challenges. However, two pivotal frameworks are guiding technological development: The National Strategy for Artificial Intelligence and the Principles for Responsible AI. These frameworks represent initial steps toward developing a structured approach to AI governance in enterprises.

While regulatory frameworks lay the groundwork for responsible AI development, ensuring AI security remains a challenge. Beyond policies and technical safeguards, the true test of AI security lies in the human oversight.

For all its complexity, encryption and obfuscation, the weakest link in AI security is not AI itself but the humans behind it. AI is perceived as intelligent, leading to overreliance and trust. However, AI does not "understand" security—it merely follows rules, which can be manipulated.    

Read More Read Less Securing AI in an unregulated landscape\r\nOrganizations must prioritize proactive security measures to mitigate LLM security risks. This requires a multi-layered approach focused on security and ethics. Key measures include:

\r\nTechnical safeguards:

\r\n\r\nInput sanitization is essential for avoiding the influence of harmful or unsuitable content on the LLM's outputs.\r\nSecuring data through encryption, both when it is stored and during transmission, will block unauthorized entities from accessing it.\r\nContinuous monitoring, logging, and testing against potential threats will promptly identify and address any misuse or security gaps.\r\n\r\nAwareness and education:

\r\n\r\nTo recognize, be aware and stay updated with the latest LLM security tools which security firms are making use of to reduce the potential dangers associated with AI. AI-driven companies are implementing advanced filtering mechanisms to monitor both inputs and outputs, ensuring that models do not generate harmful or unethical content.\r\nAI literacy is critical to the responsible adoption of security practices. Organizations should invest in educating users, employees, and stakeholders about AI’s benefits, risks and responsible use.\r\n\r\nCollaborative efforts: 

\r\n\r\nIn the absence of globally consistent GenAI regulations, industry collaboration and self-regulation become critical to share best practices, develop standardized security protocols, and create ethical guidelines for LLM deployment.\r\nA collective approach ensures that businesses align their security efforts and work together to mitigate AI-powered cyber threats, ultimately fostering responsible AI usage across sectors.\r\n\r\n"}}" id="rich-text-a4208ab19962" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Securing AI in an unregulated landscape Organizations must prioritize proactive security measures to mitigate LLM security risks. This requires a multi-layered approach focused on security and ethics. Key measures include:

Technical safeguards:

Input sanitization is essential for avoiding the influence of harmful or unsuitable content on the LLM's outputs.Securing data through encryption, both when it is stored and during transmission, will block unauthorized entities from accessing it.Continuous monitoring, logging, and testing against potential threats will promptly identify and address any misuse or security gaps. Awareness and education:

To recognize, be aware and stay updated with the latest LLM security tools which security firms are making use of to reduce the potential dangers associated with AI. AI-driven companies are implementing advanced filtering mechanisms to monitor both inputs and outputs, ensuring that models do not generate harmful or unethical content.AI literacy is critical to the responsible adoption of security practices. Organizations should invest in educating users, employees, and stakeholders about AI’s benefits, risks and responsible use. Collaborative efforts: 

In the absence of globally consistent GenAI regulations, industry collaboration and self-regulation become critical to share best practices, develop standardized security protocols, and create ethical guidelines for LLM deployment.A collective approach ensures that businesses align their security efforts and work together to mitigate AI-powered cyber threats, ultimately fostering responsible AI usage across sectors. Read More Read Less Conclusion\r\nAs the vulnerabilities of LLM systems become more evident and security breaches grow more dangerous, organizations must establish a robust and secure AI framework like ISO 42001. Responsible AI adoption is not just about innovation—it is about ensuring security, compliance and ethical deployment in an increasingly AI-driven world.

\r\n"}}" id="rich-text-125587ae287f" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Conclusion As the vulnerabilities of LLM systems become more evident and security breaches grow more dangerous, organizations must establish a robust and secure AI framework like ISO 42001. Responsible AI adoption is not just about innovation—it is about ensuring security, compliance and ethical deployment in an increasingly AI-driven world.

Read More Read Less Show article references Hide article references ‘Global Large Language Model (LLM) Market Size, Share, Growth Drivers, Competitive Analysis, Recent Trends & Developments, and Demand Forecast To 2030’, Pragma Market Research (Link)‘Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents’ 2024 by Yuyou Gan et al. (link)‘More Than 80% of Enterprises Will Have Used Generative AI APIs or Deployed Generative AI-Enabled Applications by 2026’, Gartner (Link)‘Artificial Intelligence Market to Grow at 36.6% CAGR to Garner $1,811.75 Billion by 2030’ - Grand View Research, Inc. (Link)A Comprehensive Guide to LLM Development in 2025, Turing (Link)Generative AI: Impact on Email Cyber-Attacks, Darktrace (Link)New Gemini for Workspace Vulnerability Enabling Phishing & Content Manipulation, Hidden Layer (Link)
Summary\r\nLLMs are reshaping the digital landscape, driving innovation across industries. Yet, rapid progress raises security and ethical concerns. With limited regulation, organizations must adopt proactive security measures, foster AI awareness, and collaborate on industry standards to harness AI responsibly and mitigate emerging threats.

\r\n"}}" id="rich-text-9f59b5620a03" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Summary LLMs are reshaping the digital landscape, driving innovation across industries. Yet, rapid progress raises security and ethical concerns. With limited regulation, organizations must adopt proactive security measures, foster AI awareness, and collaborate on industry standards to harness AI responsibly and mitigate emerging threats.

Read More Read Less

How EY can help

Artificial Intelligence Consulting Services

Our Consulting approach to the adoption of AI and intelligent automation is human-centered, pragmatic, outcomes-focused and ethical.

Read more

Cybersecurity Transformation

Discover how EY's Cybersecurity Transformation solution can help your organization design, deliver, and maintain cybersecurity programs.

Read more

Technology transformation

Drive business growth with technology transformation solutions at EY - integrating tech, operations & architecture to build agile, data-driven organizations

Read more

Related articles

How GenAI can drive innovation in contract management and deliver value

Discover how Generative AI can revolutionize contract management, boosting efficiency, reducing risks, and unlocking new value.

Gaurav Sharma

How AI is activating step changes in Indian education

AI in Indian education is driving step changes in the learning landscape. Explores AI tools, future trends, and innovations in the Indian education system.

Dr. Avantika Tomar

How much productivity can GenAI unlock in India? The AIdea of India 2025

Explore AIdea of India 2025 to find out how Gen AI is transforming industries, boosting productivity & reshaping India’s digital economy with innovation.

Mahesh Makhija + 3 Previous Next

About this article

Mubin Shaikh Partner, Cybersecurity Consulting, EY India With over 20 years of experience across Cybersecurity, technology risk, business continuity risk, and third-party risk management, Mubin is enabling stakeholders formulate cybersecurity strategies. Related topics AI Cybersecurity Facebook Twitter LinkedIn "> Link Copied EY Footer logo \r\nConnect with us
\r\n\r\nOur locations\r\nMy EY\r\nSite map\r\nLegal and privacy\r\n\r\n"}}" id="rich-text-26f66b56f92d" data-up-is="rich-text" data-up-translation-read-more="Read More" data-up-translation-read-less="Read Less" data-up-translation-aria-label-read-more="Read more button, press enter to activate, or use Up arrow key to learn more about this content" data-up-translation-aria-label-read-less="Read less button, press enter to activate, or use Up arrow key to learn more about this content" class="up-rich-text cmp-text" data-up-analytics="rich-text"> Connect with us
Our locationsMy EYSite mapLegal and privacy Read More Read Less Open Facebook profile Open X profile Open LinkedIn profile Open Youtube profile EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.

\r\n"}}" class="cmp-text"> EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients.

Welcome to EY in (en) You are visiting EY in (en) in en