#FactCheck-AI-Generated Video Falsely Shows CRPF Trooper Criticizing PM Modi and BJP Government

In the vast, uncharted territories of the digital world, a sinister phenomenon is proliferating at an alarming rate. It's a world where artificial intelligence (AI) and human vulnerability intertwine in a disturbing combination, creating a shadowy realm of non-consensual pornography. This is the world of deepfake pornography, a burgeoning industry that is as lucrative as it is unsettling.

According to a recent assessment, at least 100,000 deepfake porn videos are readily available on the internet, with hundreds, if not thousands, being uploaded daily. This staggering statistic prompts a chilling question: what is driving the creation of such a vast number of fakes? Is it merely for amusement, or is there a more sinister motive at play?

Recent Trends and Developments 

An investigation by India Today’s Open-Source Intelligence (OSINT) team reveals that deepfake pornography is rapidly morphing into a thriving business. AI enthusiasts, creators, and experts are extending their expertise, investors are injecting money, and even small financial companies to tech giants like Google, VISA, Mastercard, and PayPal are being misused in this dark trade. Synthetic porn has existed for years, but advances in AI and the increasing availability of technology have made it easier—and more profitable—to create and distribute non-consensual sexually explicit material. The 2023 State of Deepfake report by Home Security Heroes reveals a staggering 550% increase in the number of deepfakes compared to 2019. 

What’s the Matter with Fakes? 

But why should we be concerned about these fakes? The answer lies in the real-world harm they cause. India has already seen cases of extortion carried out by exploiting deepfake technology. An elderly man in UP’s Ghaziabad, for instance, was tricked into paying Rs 74,000 after receiving a deep fake video of a police officer. The situation could have been even more serious if the perpetrators had decided to create deepfake porn of the victim.

The danger is particularly severe for women. The 2023 State of Deepfake Report estimates that at least 98 percent of all deepfakes is porn and 99 percent of its victims are women. A study by Harvard University refrained from using the term “pornography” for creating, sharing, or threatening to create/share sexually explicit images and videos of a person without their consent. “It is abuse and should be understood as such,” it states.

Based on interviews of victims of deepfake porn last year, the study said 63 percent of participants talked about experiences of “sexual deepfake abuse” and reported that their sexual deepfakes had been monetised online. It also found “sexual deepfake abuse to be particularly harmful because of the fluidity and co-occurrence of online offline experiences of abuse, resulting in endless reverberations of abuse in which every aspect of the victim’s life is permanently disrupted”.

Creating deepfake porn is disturbingly easy. There are largely two types of deepfakes: one featuring faces of humans and another featuring computer-generated hyper-realistic faces of non-existing people. The first category is particularly concerning and is created by superimposing faces of real people on existing pornographic images and videos—a task made simple and easy by AI tools.

During the investigation, platforms hosting deepfake porn of stars like Jennifer Lawrence, Emma Stone, Jennifer Aniston, Aishwarya Rai, Rashmika Mandanna to TV actors and influencers like Aanchal Khurana, Ahsaas Channa, and Sonam Bajwa and Anveshi Jain were encountered. It takes a few minutes and as little as Rs 40 for a user to create a high-quality fake porn video of 15 seconds on platforms like FakeApp and FaceSwap.

The Modus Operandi 

These platforms brazenly flaunt their business association and hide behind frivolous declarations such as: the content is “meant solely for entertainment” and “not intended to harm or humiliate anyone”. However, the irony of these disclaimers is not lost on anyone, especially when they host thousands of non-consensual deepfake pornography.

As fake porn content and its consumers surge, deepfake porn sites are rushing to forge collaborations with generative AI service providers and have integrated their interfaces for enhanced interoperability. The promise and potential of making quick bucks have given birth to step-by-step guides, video tutorials, and websites that offer tools and programs, recommendations, and ratings.

Nearly 90 per cent of all deepfake porn is hosted by dedicated platforms that charge for long-duration premium fake content and for creating porn—of whoever a user wants, and take requests for celebrities. To encourage them further, they enable creators to monetize their content.

One such website, Civitai, has a system in place that pays “rewards” to creators of AI models that generate “images of real people'', including ordinary people. It also enables users to post AI images, prompts, model data, and LoRA (low-rank adaptation of large language models) files used in generating the images. Model data designed for adult content is gaining great popularity on the platform, and they are not only targeting celebrities. Common people are equally susceptible.

Access to premium fake porn, like any other content, requires payment. But how can a gateway process payment for sexual content that lacks consent? It seems financial institutes and banks are not paying much attention to this legal question. During the investigation, many such websites accepting payments through services like VISA, Mastercard, and Stripe were found.

Those who have failed to register/partner with these fintech giants have found a way out. While some direct users to third-party sites, others use personal PayPal accounts to manually collect money in the personal accounts of their employees/stakeholders, which potentially violates the platform's terms of use that ban the sale of “sexually oriented digital goods or content delivered through a digital medium.”

 Among others, the MakeNude.ai web app – which lets users “view any girl without clothing” in “just a single click” – has an interesting method of circumventing restrictions around the sale of non-consensual pornography. The platform has partnered with Ukraine-based Monobank and Dublin’s BetaTransfer Kassa which operates in “high-risk markets”.

BetaTransfer Kassa admits to serving “clients who have already contacted payment aggregators and received a refusal to accept payments, or aggregators stopped payments altogether after the resource was approved or completely freeze your funds”. To make payment processing easy, MakeNude.ai seems to be exploiting the donation ‘jar’ facility of Monobank, which is often used by people to donate money to Ukraine to support it in the war against Russia.

The Indian Scenario 

India currently is on its way to design dedicated legislation to address issues arising out of deepfakes. Though existing general laws requiring such platforms to remove offensive content also apply to deepfake porn. However, persecution of the offender and their conviction is extremely difficult for law enforcement agencies as it is a boundaryless crime and sometimes involves several countries in the process.

A victim can register a police complaint under provisions of Section 66E and Section 66D of the IT Act, 2000. Recently enacted Digital Personal Data Protection Act, 2023 aims to protect the digital personal data of users. Recently Union Government issued an advisory to social media intermediaries to identify misinformation and deepfakes. Comprehensive law promised by Union IT minister Ashwini Vaishnav will be able to address these challenges.

Conclusion 

In the end, the unsettling dance of AI and human vulnerability continues in the dark web of deepfake pornography. It's a dance that is as disturbing as it is fascinating, a dance that raises questions about the ethical use of technology, the protection of individual rights, and the responsibility of financial institutions. It's a dance that we must all be aware of, for it is a dance that affects us all.

References

• https://www.indiatoday.in/india/story/deepfake-porn-artificial-intelligence-women-fake-photos-2471855-2023-12-04
• https://www.hindustantimes.com/opinion/the-legal-net-to-trap-peddlers-of-deepfakes-101701520933515.html
• https://indianexpress.com/article/opinion/columns/with-deepfakes-getting-better-and-more-alarming-seeing-is-no-longer-believing/

Introduction

On March 12, the Ministry of Corporate Affairs (MCA) proposed the Bill to curb anti-competitive practices of tech giants through ex-ante regulation. The Draft Digital Competition Bill is to apply to ‘Core Digital Services,’ with the Central Government having the authority to update the list periodically. The proposed list in the Bill encompasses online search engines, online social networking services, video-sharing platforms, interpersonal communications services, operating systems, web browsers, cloud services, advertising services, and online intermediation services. 

The primary highlight of the Digital Competition Law Report created by the Committee on Digital Competition Law presented to the Parliament in the 2nd week of March 2024 involves a recommendation to introduce new legislation called the ‘Digital Competition Act,’ intended to strike a balance between certainty and flexibility. The report identified ten anti-competitive practices relevant to digital enterprises in India. These are anti-steering, platform neutrality/self-preferencing, bundling and tying, data usage (use of non-public data), pricing/ deep discounting, exclusive tie-ups, search and ranking preferencing, restricting third-party applications and finally advertising Policies.

Key Take-Aways: Digital Competition Bill, 2024

1. Qualitative and quantitative criteria for identifying Systematically Significant Digital Enterprises, if it meets any of the specified thresholds. 

• Financial thresholds in each of the immediately preceding three financial years like turnover in India, global turnover, gross merchandise value in India, or global market capitalization.
• User thresholds in each of the immediately preceding 3 financial years in India like the core digital service provided by the enterprise has at least 1 crore end users, or it has at least 10,000 business users.
• The Commission may make the designation based on other factors such as the size and resources of an enterprise, number of business or end users, market structure and size, scale and scope of activities of an enterprise and any other relevant factor.

2. A period of 90 days is provided to notify the CCI of qualification as an SSDE. Additionally, the enterprise must also notify the Commission of other enterprises within the group that are directly or indirectly involved in the provision of Core Digital Services, as Associate Digital Enterprises (ADE) and the qualification shall be for 3 years.
3. It prescribes obligations for SSDEs and their ADEs upon designation. The enterprise must comply with certain obligations regarding Core Digital Services, and non-compliance with the same shall result in penalties. Enterprises must not directly or indirectly prevent or restrict business users or end users from raising any issue of non-compliance with the enterprise’s obligations under the Act. 
4. Avoidance of favouritism in product offerings by SSDE,  its related parties, or third parties for the manufacture and sale of products or provision of services over those offered by third-party business users on the Core Digital Service in any manner.
5. The Commission will be having the same powers as vested to a civil court under the Code of Civil Procedure, 1908 when trying a suit.
6. Penalty for non-compliance without reasonable cause may extend to Rs 1 lakh for each day during which such non-compliance occurs (max. of Rs 10 crore). It may extend to 3 years or with a fine, which may extend to Rs 25 crore or with both. The Commission may also pass an order imposing a penalty on an enterprise (not exceeding 1% of the global turnover) in case it provides incorrect, incomplete, misleading information or fails to provide information. 

Suggestions and Recommendations

• The ex-ante model of regulation needs to be examined for the Indian scenario and studies need to be conducted on it has worked previously in different jurisdictions like the EU. 
• The Bill should be aimed at prioritising the fostering of fair competition by preventing monopolistic practices in digital markets exclusively. A clear distinction from the already existing Competition Act, 2002 in its functioning needs to be created so that there is no overlap in the regulations and double jeopardy is not created for enterprises.
• Restrictions on tying and bundling and data usage have been shown to negatively impact MSMEs that rely significantly on big tech to reduce operational costs and enhance customer outreach.
• Clear definitions of "dominant position" and "anti-competitive behaviour" are essential for effective enforcement in terms of digital competition need to be defined. 
• Encouraging innovation while safeguarding consumer data privacy in consonance with the DPDP Act should be the aim. Promoting interoperability and transparency in algorithms can prevent discriminatory practices. 
• Regular reviews and stakeholder consultations will ensure the law adapts to rapidly evolving technologies. 
• Collaboration with global antitrust bodies which is aimed at enhancing cross-border regulatory coherence and effectiveness.

Conclusion

The need for a competition law that is focused exclusively on Digital Enterprises is the need of the hour and hence the Committee recommended enacting the Digital Competition Act to enable CCI to selectively regulate large digital enterprises. The proposed legislation should be restricted to regulate only those enterprises that have a significant presence and ability to influence the Indian digital market. The impact of the law needs to be restrictive to digital enterprises and it should not encroach upon matters not influenced by the digital arena. India's proposed Digital Competition Bill aims to promote competition and fairness in the digital market by addressing anti-competitive practices and dominant position abuses prevalent in the digital business space. The Ministry of Corporate Affairs has received 41-page public feedback on the draft which is expected to be tabled next year in front of the Parliament. 

References

1. https://www.medianama.com/wp-content/uploads/2024/03/DRAFT-DIGITAL-COMPETITION-BILL-2024.pdf
2. https://prsindia.org/files/policy/policy_committee_reports/Report_Summary-Digital_Competition_Law.pdf
3. https://economictimes.indiatimes.com/tech/startups/meity-meets-india-inc-to-hear-out-digital-competition-law-concerns/articleshow/111091837.cms?from=mdr
4. https://www.mca.gov.in/bin/dms/getdocument?mds=gzGtvSkE3zIVhAuBe2pbow%253D%253D&type=open
5. https://www.barandbench.com/law-firms/view-point/digital-competition-laws-beginning-of-a-new-era
6. https://www.linkedin.com/pulse/policy-explainer-digital-competition-bill-nimisha-srivastava-lhltc/
7. https://www.lexology.com/library/detail.aspx?g=5722a078-1839-4ece-aec9-49336ff53b6c

‍

Introduction

In the digital entertainment world, OTT platforms have become highly popular and have attracted larger audiences. They offer a wide variety of entertaining content. However, there are certain concerns about depicting illicit or objectionable content on such platforms. The Indian Ministry of Information and Broadcasting (I&B) has been working on tackling issues like the availability of obscene content on online streaming platforms and other platforms. I&B Ministry has taken important steps to prevent the spread of such illicit or objectionable content.

The I&B Ministry has taken action against obscene and vulgar content on OTT platforms. A total 18 OTT platforms and several associated websites, apps, and social media handles have been blocked nationwide. The government has been in consistent talks with these platforms and issued several advisories, but they have not been adhered to. The decision was made after consultation with other ministries, domain experts, and industry bodies. The content allegedly obscene was found to depict nudity, sexual intercourse, and inappropriate sexual acts within societal contexts. The government states that it is the responsibility of platforms to ensure that content is not present in a vulgar fashion. Creativities do not necessarily mean promoting or propagating vulgar and sexual content.

Key Highlights of I&B Ministry Action against Obscene Content

On 14th March 2024, The Indian Ministry of Information and Broadcasting (I&B) announced the blocking of 18 OTT platforms, 19 Websites, 10 apps, and 57 social media handles for displaying obscene and vulgar content. Union Minister for Information & Broadcasting Shri Anurag Singh Thakur has announced the removal of 18 OTT platforms that published obscene and vulgar content, underscoring the responsibility of platforms to prevent the spread of such content. The decision was made under the Information Technology Act 2000 and in consultation with other Indian ministries and domain experts in media, entertainment, women's rights, and child rights.

List of Blocked OTT Platforms

OTT platforms that have been blocked are Dreams Films, Voovi, Yessma, Uncut Adda, Tri Flicks, X Prime, Neon X VIP, Besharams, Hunters, Rabbit, Xtramood, Nuefliks, MoodX, Mojflix, Hot Shots VIP, Fugi, Chikooflix, Prime Play.

It was highlighted that these OTT platforms, despite not being widely popular, have a significant viewership. One app has over 1 crore downloads, while two others have more than 50 lakh downloads on Google Play Store. These platforms also market their content through social media, with a combined followership of over 32 lakh.

Nature of content

The ministry reported that a significant portion of the content on social media platforms was obscene, vulgar, and demeaning, depicting nudity and sexual acts in inappropriate contexts like teacher-student relationships and incestuous family relationships. The content included sexual innuendos and prolonged pornographic scenes without any thematic or societal relevance. It was further stated that the content was found to be prima facie in violation of Section 67 and 67A of the Information & Technology Act, 2000, Section 292 of the Indian Penal Code and  Section 4 of the Indecent Representation of Women (Prohibition) Act, 1986.

Way Forward

The press release by the ministry stated that “The Government of India remains committed to fostering the growth and development of the OTT industry. Several measures have been undertaken in this regard, including the introduction of the Inaugural OTT Award for Web Series at the 54th International Film Festival of India, collaboration with OTT platforms in the media and entertainment sector, and the establishment of a light touch regulatory framework with an emphasis on self-regulation under the IT Rules, 2021.”

This shows that the Indian government is dedicated to promoting the growth of the OTT industry but within certain checks or oversight mechanisms to prevent illicit or objectionable content on such platforms.

OTT Content and Regulatory Checks

Online content streaming on OTT platforms lacks regulatory checks, unlike films, which are reviewed and certified by a government-appointed board. The government has instructed streaming services to independently review content for obscenity and violence before it is made available online. There have been repeated instances where criticism has been raised about the illicit or violative depicted content in some OTT shows. This highlights the issue of checks and balances. The government has urged self-regulation on platforms, but the repeated instances of illicit content raise societal concerns. The Ministry of I&B is keen towards promoting ethical & moral standards of content that is being hosted on online OTT platforms.

Conclusion

The Ministry of I&B has taken a step and announced the shutdown of 18 OTT platforms that were engaged in depicting illicit content. This shows that the I&B Ministry is committed to promoting ethical online content. While legislative measures are required to prevent the spread of such illicit or violative content, joint efforts by the government, industry players, and civil society are critical to ensuring a secure and responsible digital environment for all users.

References

• https://pib.gov.in/PressReleaseIframePage.aspx?PRID=2014477
• https://www.thehindu.com/news/national/centre-bans-ott-platforms-websites-and-apps-over-obscene-and-vulgar-content/article67949819.ece
• https://economictimes.indiatimes.com/news/india/ib-ministry-blocks-18-ott-platforms-for-vulgar-content/articleshow/108485880.cms?from=mdr
• https://indianexpress.com/article/entertainment/information-and-broadcasting-ministry-blocks-18-ott-platforms-for-obscene-and-vulgar-content-9213749/
• https://www.storyboard18.com/ott-news/mib-blocks-18-ott-platforms-for-showing-obscene-and-vulgar-content-26400.htm

‍