Meta’s Dilemma - Issue of Child Safety Online

Mr. Shrey Madaan

Research Associate, CyberPeace Foundation

PUBLISHED ON

Feb 29, 2024

Introduction

In the digital realm of social media, Meta Platforms, the driving force behind Facebook and Instagram, faces intense scrutiny following The Wall Street Journal's investigative report. This exploration delves deeper into critical issues surrounding child safety on these widespread platforms, unravelling algorithmic intricacies, enforcement dilemmas, and the ethical maze surrounding monetisation features. Instances of "parent-managed minor accounts" leveraging Meta's subscription tools to monetise content featuring young individuals have raised eyebrows. While skirting the line of legality, this practice prompts concerns due to its potential appeal to adults and the associated inappropriate interactions. It's a nuanced issue demanding nuanced solutions.

Failed Algorithms

The very heartbeat of Meta's digital ecosystem, its algorithms, has come under intense scrutiny. These algorithms, designed to curate and deliver content, were found to actively promoting accounts featuring explicit content to users with known pedophilic interests. The revelation sparks a crucial conversation about the ethical responsibilities tied to the algorithms shaping our digital experiences. Striking the right balance between personalised content delivery and safeguarding users is a delicate task.

While algorithms play a pivotal role in tailoring content to users' preferences, Meta needs to reevaluate the algorithms to ensure they don't inadvertently promote inappropriate content. Stricter checks and balances within the algorithmic framework can help prevent the inadvertent amplification of content that may exploit or endanger minors.

Major Enforcement Challenges

Meta's enforcement challenges have come to light as previously banned parent-run accounts resurrect, gaining official verification and accumulating large followings. The struggle to remove associated backup profiles adds layers to concerns about the effectiveness of Meta's enforcement mechanisms. It underscores the need for a robust system capable of swift and thorough actions against policy violators.

To enhance enforcement mechanisms, Meta should invest in advanced content detection tools and employ a dedicated team for consistent monitoring. This proactive approach can mitigate the risks associated with inappropriate content and reinforce a safer online environment for all users.

The financial dynamics of Meta's ecosystem expose concerns about the exploitation of videos that are eligible for cash gifts from followers. The decision to expand the subscription feature before implementing adequate safety measures poses ethical questions. Prioritising financial gains over user safety risks tarnishing the platform's reputation and trustworthiness. A re-evaluation of this strategy is crucial for maintaining a healthy and secure online environment.

To address safety concerns tied to monetisation features, Meta should consider implementing stricter eligibility criteria for content creators. Verifying the legitimacy and appropriateness of content before allowing it to be monetised can act as a preventive measure against the exploitation of the system.

Meta's Response

In the aftermath of the revelations, Meta's spokesperson, Andy Stone, took centre stage to defend the company's actions. Stone emphasised ongoing efforts to enhance safety measures, asserting Meta's commitment to rectifying the situation. However, critics argue that Meta's response lacks the decisive actions required to align with industry standards observed on other platforms. The debate continues over the delicate balance between user safety and the pursuit of financial gain. A more transparent and accountable approach to addressing these concerns is imperative.

To rebuild trust and credibility, Meta needs to implement concrete and visible changes. This includes transparent communication about the steps taken to address the identified issues, continuous updates on progress, and a commitment to a user-centric approach that prioritises safety over financial interests.

The formation of a task force in June 2023 was a commendable step to tackle child sexualisation on the platform. However, the effectiveness of these efforts remains limited. Persistent challenges in detecting and preventing potential child safety hazards underscore the need for continuous improvement. Legislative scrutiny adds an extra layer of pressure, emphasising the urgency for Meta to enhance its strategies for user protection.

To overcome ongoing challenges, Meta should collaborate with external child safety organisations, experts, and regulators. Open dialogues and partnerships can provide valuable insights and recommendations, fostering a collaborative approach to creating a safer online environment.

Drawing a parallel with competitors such as Patreon and OnlyFans reveals stark differences in child safety practices. While Meta grapples with its challenges, these platforms maintain stringent policies against certain content involving minors. This comparison underscores the need for universal industry standards to safeguard minors effectively. Collaborative efforts within the industry to establish and adhere to such standards can contribute to a safer digital environment for all.

To align with industry standards, Meta should actively participate in cross-industry collaborations and adopt best practices from platforms with successful child safety measures. This collaborative approach ensures a unified effort to protect users across various digital platforms.

Conclusion

Navigating the intricate landscape of child safety concerns on Meta Platforms demands a nuanced and comprehensive approach. The identified algorithmic failures, enforcement challenges, and controversies surrounding monetisation features underscore the urgency for Meta to reassess and fortify its commitment to being a responsible digital space. As the platform faces this critical examination, it has an opportunity to not only rectify the existing issues but to set a precedent for ethical and secure social media engagement.

This comprehensive exploration aims not only to shed light on the existing issues but also to provide a roadmap for Meta Platforms to evolve into a safer and more responsible digital space. The responsibility lies not just in acknowledging shortcomings but in actively working towards solutions that prioritise the well-being of its users.

References

‍

PUBLISHED ON

Feb 29, 2024

Related Blogs

Supreme Court Stay on Centre's Notification of PIB's Fact Check Unit Under IT Amendment Rules 2023

March 23, 2024

Introduction

On 20th March 2024, the Indian government notified the Fact Check Unit (FCU) under the Press Information Bureau (PIB) of the Ministry of Information and Broadcasting as the Fact Check Unit (FCU) of the Central Government. This PIB FCU is notified under the provisions of Rule 3(1)(b)(v) of the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules 2023 (IT Amendment Rules 2023).

However, the next day, on 21st March 2024, the Supreme Court stayed the Centre's decision. The IT Amendment Rules of 2023 provide that the Ministry of Electronics and Information Technology (MeitY) can notify a fact-checking body to identify and tag what it considers fake news with respect to any activity of the Centre. The stay will be in effect till the Bombay High Court finally decides the challenges to the IT Rules amendment 2023.

The official notification dated 20th March 2024 read as follows:

“In exercise of the powers conferred by sub-clause (v) of clause (b) of sub-rule (1) of rule 3 of the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, the Central Government hereby notifies the Fact Check Unit under the Press Information Bureau of the Ministry of Information and Broadcasting as the fact check unit of the Central Government for the purposes of the said sub-clause, in respect of any business of the Central Government.”

Impact of the notification

The impact of notifying PIB’s FCU under Rule 3(1)(b)(v)will empower the PIB’s FCU to issue direct takedown directions to the concerned Intermediary. Any information posted on social media in relation to the business of the central government that has been flagged as fake or false by the FCU has to be taken down by the concerned intermediary. If it fails to do so, it will lose the 'safe harbour' immunity against legal proceedings arising out of such information posted offered under Section 79 of IT Act, 2000.

Safe harbour provision u/s 79 of IT Act, 2000

Section 79 of the IT Act, 2000 serves as a safe harbour provision for intermediaries. The provision states that "an intermediary shall not be liable for any third-party information, data, or communication link made available or hosted by him". However, it is notable that this legal immunity cannot be granted if the intermediary "fails to expeditiously" take down a post or remove a particular content after the government or its agencies flag that the information is being used unlawfully. Furthermore, intermediaries are obliged to observe due diligence on their platforms.

Rule 3 (1)(b)(v) Under IT Amendment Rules 2023

Rule 3(1)(b)(v) of The Information Technology(Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 [updated as on 6.4.2023] provides that all intermediaries [Including a social media intermediary, a significant social media intermediary and an online gaming intermediary], are required to make "reasonable efforts” or perform due diligence to ensure that their users do not "host, display, upload, modify, publish, transmit, store, update or share” any information that “deceives or misleads the addressee about the origin of the message or knowingly and intentionally communicates any misinformation or information which is patently false and untrue or misleading in nature or, in respect of any business of the Central Government, is identified as fake or false or misleading by such fact check unit of the Central Government as the Ministry may, by notification published in the Official Gazette, specify”.

PIB - FCU

The PIB - Fact Check Unit(FCU) was established in November 2019 to prevent the spread of fake news and misinformation about the Indian government. It also provides an accessible platform for people to report suspicious or questionable information related to the Indian government. This FCU is responsible for countering misinformation on government policies, initiatives, and schemes. The FCU is tasked with addressing misinformation about government policies, initiatives, and schemes, either directly (Suo moto) or through complaints received. On 20th March 2024,via a gazetted notification, the Centre notified the Press Information Bureau's fact-check unit (FCU) as the nodal agency to flag fake news or misinformation related to the central government. However, The Supreme Court stayed the Centre's notification of the Fact-Check Unit under IT Amendment Rules 2023.

Concerns with IT Amendment Rules 2023

The Ministry of Electronics and Information Technology(MeitY) amended the IT Rules of 2021. The ‘Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2023’ (IT Amendment Rules 2023) were notified by the Ministry of Electronics and Information Technology on 6 April 2023. The rules introduced new provisions to establish a fact-checking unit with respect to “any business of the central government” and also made other provisions pertaining to online gaming.

The Constitutional validity of IT Amendment Rules 2023 has been challenged through a writ petition challenging the IT Rules 2023 in the Bombay High Court. The contention is that the rules raise "serious constitutional questions," and Rule 3(1)(b)(v), as amended in 2023, impacts the fundamental right to freedom of speech and expression would fall for analysis by the High Court.

Supreme Court Stays Setting up of FCU

A bench comprising Chief Justice DY Chandra Hud, Justices JB Pardiwala and Manoj Misra convened to hear Special Leave Petitions filed by Kunal Kamra, the Editors Guild of India and the Association of Indian Magazines challenging the refusal of the Bombay High Court to stay the implementation of the IT Rules 2023. The Supreme Court has stayed the Union's notification of the Fact-Check Unit under the IT Amendment Rules 2023, pending the Bombay High Court's decision on the challenges to the IT Rules Amendment 2023.

Emphasizing Freedom of Speech in the Democratic Environment

The advent of advanced technology has also brought with it a new generation of threats and concerns: the misuse of said technology in the form of deepfakes and misinformation is one of the most pressing concerns plaguing society today. This realization has informed the critical need for stringent regulatory measures. The government is rightly prioritizing the need to immediately address digital threats, but there must be a balance between our digital security policies and the need to respect free speech and critical thinking. The culture of open dialogue is the bedrock of democracy. The ultimate truth is shaped through free trade in ideas within a competitive marketplace of ideas. The constitutional scheme of democracy places great importance on the fundamental value of liberty of thought and expression, which has also been emphasized by the Supreme Court in its various judgements.

The IT Rules, 2023,provide for creating a "fact check unit" to identify fake or false or misleading information “in relation to any business of the central government "This move raised concerns within the media fraternity, who argued that the determination of fake news cannot be placed solely in the hands of the government. It is also worth noting that if users post something illegal, they can still be punished under laws that already exist in the country.

We must take into account that freedom of speech under Article 19 of the Constitution is not an absolute right. Article 19(2) imposes restrictions on the Right to Freedom of Speech and expression. Hence, there has to be a balance between regulatory measures and citizens' fundamental rights.

Nowadays, the term ‘fake news’ is used very loosely. Additionally, there is a dearth of clearly established legal parameters that define what amounts to fake or misleading information. Clear definitions of the terms should be established to facilitate certainty as to what content is ‘fake news’ and what content is not. Any such restriction on speech must align with the exceptions outlined in Article19(2) of the Constitution.

Conclusion

Through a government notification, PIB - FCU was intended to act as a government-run fact-checking body to verify any information about the Central Government. However, the apex court of India stayed the Centre's notification. Now, the matter is sub judice, and we hope for the judicial analysis of the validity of IT Amendment Rules 2023.

Notably, the government is implementing measures to combat misinformation in the digital world, but it is imperative that we strive for a balance between regulatory checks and individual rights. As misinformation spreads across all sectors, a centralised approach is needed in order to tackle it effectively. Regulatory reforms must take into account the crucial roleplayed by social media in today’s business market: a huge amount of trade and commerce takes place online or is informed by digital content, which means that the government must introduce policies and mechanisms that continue to support economic activity. Collaborative efforts between the government and its agencies, technological companies, and advocacy groups are needed to deal with the issue better at a higher level.

References

‍

MeitY’s Efforts in Combatting Deepfakes

December 21, 2024

Introduction:

As the Ministry of Electronic and Information Technology (MeitY) continues to invite proposals from academicians, institutions, and industry experts to develop frameworks and tools for AI-related issues through the IndiaAI Mission, it has also funded two AI projects that will deal with matters related to deepfakes as per a status report submitted on 21st November 2024. The Delhi court also ordered the nomination of the members of a nine-member Committee constituted by the MeitY on 20th November 2024 (to address deepfake issues) and asked for a report within three months.

Funded AI projects

‍

The two projects funded by MeitY are :

Fake Speech Detection Using Deep Learning Framework- The project was initiated in December 2021 and focuses on detecting fake speech by creating a web interface for detection software this also includes investing in creating a speech verification software platform that is specifically designed for testing fake speech detection systems. It is set to end in December 2024.
Design and Development of Software for Detecting Deepfake Videos and Images- This project was funded by MeitY from January 2022 to March 2024. It also involved the Centre for Development of Advanced Computing (C-DAC), Kolkata and Hyderabad as they have developed a prototype tool capable of detecting deepfakes. Named FakeCheck, it is designed as a desktop application and a web portal aiming to detect deepfakes without the use of the internet. Reports suggest that it is currently undergoing the testing phase and awaiting feedback.

‍

Apart from these projects, MeitY has released their expression of interest for proposals in four other areas which include:

Tools that detect AI-generated content along with traceable markers,
Tools that develop an ethical AI framework for AI systems to be transparent and respect human values,
An AI risk management and assessment tool that analyses threats and precarious situations of AI-specific risks in public AI use cases and; hi very well so thank you Hauer you thanks
Tools that can assess the resilience of AI in stressful situations such as cyberattacks, national disasters, operational failures, etc.

CyberPeace Outlook

Deepfakes pose significant challenges to critical sectors in India, such as healthcare and education, where manipulated content can lead to crimes like digital impersonation, misinformation, and fraud. The rapid advancement of AI, with developments (regarding regulation) that can’t keep pace, continues to fuel such threats. Recognising these risks, MeitY’s IndiaAI mission, promoting investments and encouraging educational institutions to undertake AI projects that strengthen the country's digital infrastructure comes in as a guiding light. A part of the mission focuses on developing indigenous solutions, including tools for assessment and regulation, to address AI-related threats effectively. While India is making strides in this direction, the global AI landscape is evolving rapidly, with many nations advancing regulations to mitigate AI-driven challenges. Consistent steps, including inviting proposals and funding projects provide the much-needed impetus for the mission to be realised.

References

‍

Navigating the Treacherous Terrain of Online Romance Scams in India

February 23, 2024

Introduction

Romance scams have been rised in India. A staggering 66 percent of individuals in India have been ensnared by the siren songs of deceitful online dating schemes. These are not the attempts of yesteryears but rather a new breed of scams, seamlessly weaving the threads of traditional deceit with the sinew of cutting-edge technologies such as generative AI and deep fakes. A report by Tenable highlights the rise of romance scams in India, which now combine traditional tactics with advanced technologies like generative AI and deepfakes. Over 69% of Indians struggle to distinguish between artificial and authentic human voices. Scammers are using celebrity impersonations and platforms like Facebook to lure victims into a false sense of security.

The Romance Scam

A report by Tenable, the exposure management company, illuminates the disturbing evolution of these romance scams. It reveals a reality: AI-generated deep lakes have attained a level of sophistication where an astonishing 69 percent of Indians confess to struggling to discern between artificial and authentic human voices. This technological prowess has armed scammers with the tools to craft increasingly convincing personas, enabling them to perpetrate their nefarious acts with alarming success.

In 2023 alone, 43 percent of Indians reported falling victim to AI voice scams, with a staggering 83 percent of those targeted suffering financial loss. The scammers, like puppeteers, manipulate their digital marionettes with a deftness that is both awe-inspiring and horrifying. They have mastered the art of impersonating celebrities and fabricating personas that resonate with their targets, particularly preying on older demographics who may be more susceptible to their charms.

Social media platforms, which were once heralded as the town squares of the 21st century, have unwittingly become fertile grounds for these fraudulent activities. They lure victims into a false sense of security before the scammers orchestrate their deceitful symphonies. Chris Boyd, a staff research engineer at Tenable, issues a stern warning against the lure of private conversations, where the protective layers of security are peeled away, leaving individuals exposed to the machinations of these digital charlatans.

The Vulnerability of Individuals

The report highlights the vulnerability of certain individuals, especially those who are older, widowed, or experiencing memory loss. These individuals are systematically targeted by heartless criminals who exploit their longing for connection and companionship. The importance of scrutinising requests for money from newfound connections is underscored, as is the need for meticulous examination of photographs and videos for any signs of manipulation or deceit.

'Increasing awareness and maintaining vigilance are our strongest weapons against these heartless manipulations, 'safeguarding love seekers from the treacherous web of AI-enhanced deception.'

The landscape of love has been irrevocably altered by the prevalence of smartphones and the deep proliferation of mobile internet. Finding love has morphed into a digital odyssey, with more and more Indians turning to dating apps like Tinder, Bumble, and Hinge. Yet, as with all technological advancements, there lurks a shadowy underbelly. The rapid adoption of dating sites has provided potential scammers with a veritable goldmine of opportunity.

It is not uncommon these days to hear tales of individuals who have lost their life savings to a person they met on a dating site or who have been honey-trapped and extorted by scammers on such platforms. A new study, titled 'Modern Love' and published by McAfee ahead of Valentine's Day 2024, reveals that such scams are rampant in India, with 39 percent of users reporting that their conversations with a potential love interest online turned out to be with a scammer.

The study also found that 77 percent of Indians have encountered fake profiles and photos that appear AI-generated on dating websites or apps or on social media, while 26 percent later discovered that they were engaging with AI-generated bots rather than real people. 'The possibilities of AI are endless, and unfortunately, so are the perils,' says Steve Grobman, McAfee’s Chief Technology Officer.

Steps to Safeguard

Scammers have not limited their hunting grounds to dating sites alone. A staggering 91 percent of Indians surveyed for the study reported that they, or someone they know, have been contacted by a stranger through social media or text message and began to 'chat' with them regularly. Cybercriminals exploit the vulnerability of those seeking love, engaging in long and sophisticated attempts to defraud their victims.

McAfee offers some steps to protect oneself from online romance and AI scams:

Scrutinise any direct messages you receive from a love interest via a dating app or social media.
Be on the lookout for consistent, AI-generated messages which often lack substance or feel generic.
Avoid clicking on any links in messages from someone you have not met in person.
Perform a reverse image search of any profile pictures used by the person.
Refrain from sending money or gifts to someone you haven’t met in person, even if they send you money first.
Discuss your new love interest with your trusted friend. It can be easy to overlook red flags when you are hopeful and excited.

Conclusion

The path is fraught with illusions, and only by arming oneself with knowledge and scepticism can one hope to find true connection without falling prey to the mirage of deceit. As we navigate this treacherous terrain, let us remember that the most profound connections are often those that withstand the test of time and the scrutiny of truth.

References

‍

Meta’s Dilemma - Issue of Child Safety Online

Introduction

Failed Algorithms

Major Enforcement Challenges

Meta's Response

Conclusion

References

Related Blogs

Introduction

The official notification dated 20th March 2024 read as follows:

Impact of the notification

Safe harbour provision u/s 79 of IT Act, 2000

Rule 3 (1)(b)(v) Under IT Amendment Rules 2023

PIB - FCU

Concerns with IT Amendment Rules 2023

Supreme Court Stays Setting up of FCU

Emphasizing Freedom of Speech in the Democratic Environment

Conclusion

References

Introduction:

Funded AI projects

CyberPeace Outlook

References

Introduction

The Romance Scam

The Vulnerability of Individuals

Steps to Safeguard

McAfee offers some steps to protect oneself from online romance and AI scams:

Conclusion

References

Become a part of our vision to make the digital world safe for all!

Awareness

Engagement

Play your part for CyberPeace