#FactCheck-AI-Generated Graphic Falsely Claims Union Education Minister Dharmendra Pradhan Will Resign on July 22.

Introduction

Romance scams have been rised in India. A staggering 66 percent of individuals in India have been ensnared by the siren songs of deceitful online dating schemes. These are not the attempts of yesteryears but rather a new breed of scams, seamlessly weaving the threads of traditional deceit with the sinew of cutting-edge technologies such as generative AI and deep fakes. A report by Tenable highlights the rise of romance scams in India, which now combine traditional tactics with advanced technologies like generative AI and deepfakes. Over 69% of Indians struggle to distinguish between artificial and authentic human voices. Scammers are using celebrity impersonations and platforms like Facebook to lure victims into a false sense of security.

The Romance Scam

A report by Tenable, the exposure management company, illuminates the disturbing evolution of these romance scams. It reveals a reality: AI-generated deep lakes have attained a level of sophistication where an astonishing 69 percent of Indians confess to struggling to discern between artificial and authentic human voices. This technological prowess has armed scammers with the tools to craft increasingly convincing personas, enabling them to perpetrate their nefarious acts with alarming success.

In 2023 alone, 43 percent of Indians reported falling victim to AI voice scams, with a staggering 83 percent of those targeted suffering financial loss. The scammers, like puppeteers, manipulate their digital marionettes with a deftness that is both awe-inspiring and horrifying. They have mastered the art of impersonating celebrities and fabricating personas that resonate with their targets, particularly preying on older demographics who may be more susceptible to their charms.

Social media platforms, which were once heralded as the town squares of the 21st century, have unwittingly become fertile grounds for these fraudulent activities. They lure victims into a false sense of security before the scammers orchestrate their deceitful symphonies. Chris Boyd, a staff research engineer at Tenable, issues a stern warning against the lure of private conversations, where the protective layers of security are peeled away, leaving individuals exposed to the machinations of these digital charlatans.

The Vulnerability of Individuals 

The report highlights the vulnerability of certain individuals, especially those who are older, widowed, or experiencing memory loss. These individuals are systematically targeted by heartless criminals who exploit their longing for connection and companionship. The importance of scrutinising requests for money from newfound connections is underscored, as is the need for meticulous examination of photographs and videos for any signs of manipulation or deceit.

'Increasing awareness and maintaining vigilance are our strongest weapons against these heartless manipulations, 'safeguarding love seekers from the treacherous web of AI-enhanced deception.'

The landscape of love has been irrevocably altered by the prevalence of smartphones and the deep proliferation of mobile internet. Finding love has morphed into a digital odyssey, with more and more Indians turning to dating apps like Tinder, Bumble, and Hinge. Yet, as with all technological advancements, there lurks a shadowy underbelly. The rapid adoption of dating sites has provided potential scammers with a veritable goldmine of opportunity.

It is not uncommon these days to hear tales of individuals who have lost their life savings to a person they met on a dating site or who have been honey-trapped and extorted by scammers on such platforms. A new study, titled 'Modern Love' and published by McAfee ahead of Valentine's Day 2024, reveals that such scams are rampant in India, with 39 percent of users reporting that their conversations with a potential love interest online turned out to be with a scammer.

The study also found that 77 percent of Indians have encountered fake profiles and photos that appear AI-generated on dating websites or apps or on social media, while 26 percent later discovered that they were engaging with AI-generated bots rather than real people. 'The possibilities of AI are endless, and unfortunately, so are the perils,' says Steve Grobman, McAfee’s Chief Technology Officer.

Steps to Safeguard 

Scammers have not limited their hunting grounds to dating sites alone. A staggering 91 percent of Indians surveyed for the study reported that they, or someone they know, have been contacted by a stranger through social media or text message and began to 'chat' with them regularly. Cybercriminals exploit the vulnerability of those seeking love, engaging in long and sophisticated attempts to defraud their victims.

McAfee offers some steps to protect oneself from online romance and AI scams:

• Scrutinise any direct messages you receive from a love interest via a dating app or social media.
• Be on the lookout for consistent, AI-generated messages which often lack substance or feel generic.
• Avoid clicking on any links in messages from someone you have not met in person.
• Perform a reverse image search of any profile pictures used by the person.
• Refrain from sending money or gifts to someone you haven’t met in person, even if they send you money first.
• Discuss your new love interest with your trusted friend. It can be easy to overlook red flags when you are hopeful and excited.

Conclusion

The path is fraught with illusions, and only by arming oneself with knowledge and scepticism can one hope to find true connection without falling prey to the mirage of deceit. As we navigate this treacherous terrain, let us remember that the most profound connections are often those that withstand the test of time and the scrutiny of truth.

References 

• https://www.businesstoday.in/technology/news/story/valentine-day-alert-deepfakes-genai-amplifying-romance-scams-in-india-warn-researchers-417245-2024-02-13
• https://www.indiatimes.com/amp/news/india/valentines-day-around-40-per-cent-indians-have-been-scammed-while-looking-for-love-online-627324.html
• https://zeenews.india.com/technology/valentine-day-deepfakes-in-romance-scams-generative-ai-in-scams-romance-scams-in-india-online-dating-scams-in-india-ai-voice-scams-in-india-cyber-security-in-india-2720589.html
• https://www.mcafee.com/en-us/consumer-corporate/newsroom/press-releases/2023/20230209.html

‍

The World Economic Forum reported that AI-generated misinformation and disinformation are the second most likely threat to present a material crisis on a global scale in 2024 at 53% (Sept. 2023). Artificial intelligence is automating the creation of fake news at a rate disproportionate to its fact-checking. It is spurring an explosion of web content mimicking factual articles that instead disseminate false information about grave themes such as elections, wars and natural disasters. 

According to a report by the Centre for the Study of Democratic Institutions, a Canadian think tank, the most prevalent effect of Generative AI is the ability to flood the information ecosystem with misleading and factually-incorrect content. As reported by Democracy Reporting International during the 2024 elections of the European Union, Google's Gemini, OpenAI’s ChatGPT 3.5 and 4.0, and Microsoft’s AI interface ‘CoPilot’ were inaccurate one-third of the time when engaged for any queries regarding the election data.   Therefore, a need for an innovative regulatory approach like regulatory sandboxes which can address these challenges while encouraging responsible AI innovation is desired.

What Is AI-driven Misinformation?

False or misleading information created, amplified, or spread using artificial intelligence technologies is AI-driven misinformation. Machine learning models are leveraged to automate and scale the creation of false and deceptive content. Some examples are deep fakes, AI-generated news articles, and bots that amplify false narratives on social media.

The biggest challenge is in the detection and management of AI-driven misinformation. It is difficult to distinguish AI-generated content from authentic content, especially as these technologies advance rapidly. 

AI-driven misinformation can influence elections, public health, and social stability by spreading false or misleading information. While public adoption of the technology has undoubtedly been rapid, it is yet to achieve true acceptance and actually fulfill its potential in a positive manner because there is widespread cynicism about the technology - and rightly so. The general public sentiment about AI is laced with concern and doubt regarding the technology’s trustworthiness, mainly due to the absence of a regulatory framework maturing on par with the technological development. 

Regulatory Sandboxes: An Overview

Regulatory sandboxes refer to regulatory tools that allow businesses to test and experiment with innovative products, services or businesses under the supervision of a regulator for a limited period. They engage by creating a controlled environment where regulators allow businesses to test new technologies or business models with relaxed regulations.

Regulatory sandboxes have been in use for many industries and the most recent example is their use in sectors like fintech, such as the UK’s Financial Conduct Authority sandbox. These models have been known to encourage innovation while allowing regulators to understand emerging risks. Lessons from the fintech sector show that the benefits of regulatory sandboxes include facilitating firm financing and market entry and increasing speed-to-market by reducing administrative and transaction costs. For regulators, testing in sandboxes informs policy-making and regulatory processes. Looking at the success in the fintech industry, regulatory sandboxes could be adapted to AI, particularly for overseeing technologies that have the potential to generate or spread misinformation.

The Role of Regulatory Sandboxes in Addressing AI Misinformation

Regulatory sandboxes can be used to test AI tools designed to identify or flag misinformation without the risks associated with immediate, wide-scale implementation. Stakeholders like AI developers, social media platforms, and regulators work in collaboration within the sandbox to refine the detection algorithms and evaluate their effectiveness as content moderation tools.

These sandboxes can help balance the need for innovation in AI and the necessity of protecting the public from harmful misinformation. They allow the creation of a flexible and adaptive framework capable of evolving with technological advancements and fostering transparency between AI developers and regulators. This would lead to more informed policymaking and building public trust in AI applications.

CyberPeace Policy Recommendations 

Regulatory sandboxes offer a mechanism to predict solutions that will help to regulate the misinformation that AI tech creates. Some policy recommendations are as follows: 

1. Create guidelines for a global standard for including regulatory sandboxes that can be adapted locally and are useful in ensuring consistency in tackling AI-driven misinformation.
2. Regulators can propose to offer incentives to companies that participate in sandboxes. This would encourage innovation in developing anti-misinformation tools, which could include tax breaks or grants.
3. Awareness campaigns can help in educating the public about the risks of AI-driven misinformation and the role of regulatory sandboxes can help manage public expectations.
4. Periodic and regular reviews and updates to the sandbox frameworks should be conducted to keep pace with advancements in AI technology and emerging forms of misinformation should be emphasized.

‍

Conclusion and the Challenges for Regulatory Frameworks

Regulatory sandboxes offer a promising pathway to counter the challenges that AI-driven misinformation poses while fostering innovation. By providing a controlled environment for testing new AI tools, these sandboxes can help refine technologies aimed at detecting and mitigating false information. This approach ensures that AI development aligns with societal needs and regulatory standards, fostering greater trust and transparency. With the right support and ongoing adaptations, regulatory sandboxes can become vital in countering the spread of AI-generated misinformation, paving the way for a more secure and informed digital ecosystem.

References

• https://www.thehindu.com/sci-tech/technology/on-the-importance-of-regulatory-sandboxes-in-artificial-intelligence/article68176084.ece
• https://www.oecd.org/en/publications/regulatory-sandboxes-in-artificial-intelligence_8f80a0e6-en.html
• https://www.weforum.org/publications/global-risks-report-2024/
• https://democracy-reporting.org/en/office/global/publications/chatbot-audit#Conclusions

‍

Introduction

The 2025 Delhi Legislative Assembly election is just around the corner, scheduled for February 5, 2025, with all 70 constituencies heading to the polls. The eagerly awaited results will be announced on February 8, bringing excitement as the people of Delhi prepare to see their chosen leader take the helm as Chief Minister. As the election season unfolds, social media becomes a buzzing hub of activity, with information spreading rapidly across platforms. However, this period also sees a surge in online mis/disinformation, making elections a hotspot for misleading content. It is crucial for citizens to exercise caution and remain vigilant against false or deceptive online posts, videos, or content. Empowering voters to distinguish facts from fiction and recognize the warning signs of misinformation is essential to ensure informed decision-making. By staying alert and well-informed, we can collectively safeguard the integrity of the democratic process.

‍

Risks of Mis/Disinformation

According to the 2024 survey report titled ‘Truth Be Told’ by ‘The 23 Watts’, 90% of Delhi’s youth (Gen Z) report witnessing a spike in fake news during elections, and 91% believe it influences voting patterns. Furthermore, the research highlights that 14% of Delhi’s youth tend to share sensational news without fact-checking, relying solely on conjecture.

‍

Recent Measures by ECI

Recently the Election Commission of India (EC) has issued a fresh advisory to political parties to ensure responsible use of AI-generated content in their campaigns. The EC has issued guidelines to curb the potential use of "deepfakes" and AI-generated distorted content by political parties and their representatives to disturb the level playing field. EC has mandated the labelling of all AI-generated content used in election campaigns to enhance transparency, combat misinformation, ensuring a fair electoral process in the face of rapidly advancing AI technologies.

‍

Best Practices to Avoid Electoral Mis/Disinformation 

• Seek Information from Official Sources: Voters should rely on authenticated sources for information. These include reading official manifestos, following verified advisory notifications from the Election Commission, and avoiding unverified claims or rumours.
• Consume News Responsibly: Voters must familiarize themselves with dependable news channels and make use of reputable fact-checking organizations that uphold the integrity of news content. It is crucial to refrain from randomly sharing or forwarding any news post, video, or message without verifying its authenticity. Consume responsibly, fact-check thoroughly, and share cautiously.
• Role of Fact-Checking: Cross-checking and verifying information from credible sources are indispensable practices. Reliable and trustworthy fact-checking tools are vital for assessing the authenticity of information in the digital space. Voters are encouraged to use these tools to validate information from authenticated sources and adopt a habit of verification on their own. This approach fosters a culture of critical thinking, empowering citizens to counter deceptive deepfakes and malicious misinformation effectively. It also helps create a more informed and resilient electorate.
• Be Aware of Electoral Deepfakes: In the era of artificial intelligence, synthetic media presents significant challenges. Just as videos can be manipulated, voices can also be cloned. It is essential to remain vigilant against the misuse of deepfake audio and video content by malicious actors. Recognize the warning signs, such as inconsistencies or unnatural details, and stay alert to misleading multimedia content. Proactively question and verify such material to avoid falling prey to deception.

References

1. https://www.financialexpress.com/business/brandwagon-90-ofnbsp-delhi-youth-witness-spike-in-fake-news-during-elections-91-believe-it-influences-voting-patterns-revealed-the-23-watts-report-3483166/
2. https://timesofindia.indiatimes.com/india/election-commission-urges-parties-to-disclose-ai-generated-campaign-content-in-interest-of-transparency/articleshow/117306865.cms
3. https://www.thehindu.com/news/national/election-commission-issues-advisory-on-use-of-ai-in-poll-campaigning/article69103888.ece
4. https://indiaai.gov.in/article/election-commission-of-india-embraces-ai-ethics-in-campaigning-advisory-on-labelling-ai-generated-content