#FactCheck - Old Video of US Soldiers’ Coffins Mislinked to Iran War

Introduction

In a world where Artificial Intelligence (AI) is already changing the creation and consumption of content at a breathtaking pace, distinguishing between genuine media and false or doctored content is a serious issue of international concern. AI-generated content in the form of deepfakes, synthetic text and photorealistic images is being used to disseminate misinformation, shape public opinion and commit fraud. As a response, governments, tech companies and regulatory bodies are exploring ‘watermarking’ as a key mechanism to promote transparency and accountability in AI-generated media. Watermarking embeds identifiable information into content to indicate its artificial origin. 

Government Strategies Worldwide

Governments worldwide have pursued different strategies to address AI-generated media through watermarking standards. In the US, President Biden's 2023 Executive Order on AI directed the Department of Commerce and the National Institute of Standards and Technology (NIST) to establish clear guidelines for digital watermarking of AI-generated content. This action puts a big responsibility on large technology firms to put identifiers in media produced by generative models. These identifiers should help fight misinformation and address digital trust.

The European Union, in its Artificial Intelligence Act of 2024, requires AI-generated content to be labelled. Article 50 of the Act specifically demands that developers indicate whenever users engage with synthetic content. In addition, the EU is a proponent of the Coalition for Content Provenance and Authenticity (C2PA), an organisation that produces secure metadata standards to track the origin and changes of digital content.

India is currently in the process of developing policy frameworks to address AI and synthetic content, guided by judicial decisions that are helping shape the approach. In 2024, the Delhi High Court directed the central government to appoint members for a committee responsible for regulating deepfakes. Such moves indicate the government's willingness to regulate AI-generated content. 

China, has already implemented mandatory watermarking on all deep synthesis content. Digital identifiers must be embedded in AI media by service providers, and China is one of the first countries to adopt stern watermarking legislation.

Understanding the Technical Feasibility

Watermarking AI media means inserting recognisable markers into digital material. They can be perceptible, such as logos or overlays or imperceptible, such as cryptographic tags or metadata. Sophisticated methods such as Google's SynthID apply imperceptible pixel-level changes that remain intact against standard image manipulation such as resizing or compression. Likewise, C2PA metadata standards enable the user to track the source and provenance of an item of content.

Nonetheless, watermarking is not an infallible process. Most watermarking methods are susceptible to tampering. Aforementioned adversaries with expertise, for instance, can use cropping editing or AI software to delete visible watermarks or remove metadata. Further, the absence of interoperability between different watermarking systems and platforms hampers their effectiveness. Scalability is also an issue enacting and authenticating watermarks for billions of units of online content necessitates huge computational efforts and routine policy enforcement across platforms. Scientists are currently working on solutions such as blockchain-based content authentication and zero-knowledge watermarking, which maintain authenticity without sacrificing privacy. These new techniques have potential for overcoming technical deficiencies and making watermarking more secure.

Challenges in Enforcement

Though increasing agreement exists for watermarking, implementation of such policies is still a major issue. Jurisdictional constraints prevent enforceability globally. A watermarking policy within one nation might not extend to content created or stored in another, particularly across decentralised or anonymous domains. This creates an exigency for international coordination and the development of worldwide digital trust standards. While it is a welcome step that platforms like Meta, YouTube, and TikTok have begun flagging AI-generated content, there remains a pressing need for a standardised policy that ensures consistency and accountability across all platforms. Voluntary compliance alone is insufficient without clear global mandates.

User literacy is also a significant hurdle. Even when content is properly watermarked, users might not see or comprehend its meaning. This aligns with issues of dealing with misinformation, wherein it's not sufficient just to mark off fake content, users need to be taught how to think critically about the information they're using. Public education campaigns, digital media literacy and embedding watermarking labels within user-friendly UI elements are necessary to ensure this technology is actually effective.

Balancing Privacy and Transparency

While watermarking serves to achieve digital transparency, it also presents privacy issues. In certain instances, watermarking might necessitate the embedding of metadata that will disclose the source or identity of the content producer. This threatens journalists, whistleblowers, activists, and artists utilising AI tools for creative or informative reasons. Governments have a responsibility to ensure that watermarking norms do not violate freedom of expression or facilitate surveillance. The solution is to achieve a balance by employing privacy-protection watermarking strategies that verify the origin of the content without revealing personally identifiable data. "Zero-knowledge proofs" in cryptography may assist in creating watermarking systems that guarantee authentication without undermining user anonymity.

On the transparency side, watermarking can be an effective antidote to misinformation and manipulation. For example, during the COVID-19 crisis, misinformation spread by AI on vaccines, treatments and public health interventions caused widespread impact on public behaviour and policy uptake. Watermarked content would have helped distinguish between authentic sources and manipulated media and protected public health efforts accordingly.

Best Practices and Emerging Solutions

Several programs and frameworks are at the forefront of watermarking norms. Adobe, Microsoft and others' collaborative C2PA framework puts tamper-proof metadata into images and videos, enabling complete traceability of content origin. SynthID from Google is already implemented on its Imagen text-to-image model and secretly watermarks images generated by AI without any susceptibility to tampering. The Partnership on AI (PAI) is also taking a leadership role by building out ethical standards for synthetic content, including standards around provenance and watermarking. These frameworks become guides for governments seeking to introduce equitable, effective policies. In addition, India's new legal mechanisms on misinformation and deepfake regulation present a timely point to integrate watermarking standards consistent with global practices while safeguarding civil liberties.

Conclusion

Watermarking regulations for synthetic media content are an essential step toward creating a safer and more credible digital world. As artificial media becomes increasingly indistinguishable from authentic content, the demand for transparency, origin, and responsibility increases. Governments, platforms, and civil society organisations will have to collaborate to deploy watermarking mechanisms that are technically feasible, compliant and privacy-friendly. India is especially at a turning point, with courts calling for action and regulatory agencies starting to take on the challenge. Empowering themselves with global lessons, applying best-in-class watermarking platforms and promoting public awareness can enable the nation to acquire a level of resilience against digital deception. 

References

1. https://artificialintelligenceact.eu/
2. https://www.cyberpeace.org/resources/blogs/delhi-high-court-directs-centre-to-nominate-members-for-deepfake-committee
3.  https://c2pa.org
4. https://www.cyberpeace.org/resources/blogs/misinformations-impact-on-public-health-policy-decisions
5.  https://deepmind.google/technologies/synthid/
6. https://www.imatag.com/blog/china-regulates-ai-generated-content-towards-a-new-global-standard-for-transparency

‍

AI has grown manifold in the past decade and so has its reliance. A MarketsandMarkets study estimates the AI market to reach $1,339 billion by 2030. Further, Statista reports that ChatGPT amassed more than a million users within the first five days of its release, showcasing its rapid integration into our lives. This development and integration have their risks. Consider this response from Google’s AI chatbot, Gemini to a student’s homework inquiry: “You are not special, you are not important, and you are not needed…Please die.” In other instances, AI has suggested eating rocks for minerals or adding glue to pizza sauce. Such nonsensical outputs are not just absurd; they’re dangerous. They underscore the urgent need to address the risks of unrestrained AI reliance. 

AI’s Rise and Its Limitations

The swiftness of AI’s rise, fueled by OpenAI's GPT series, has revolutionised fields like natural language processing, computer vision, and robotics. Generative AI Models like GPT-3, GPT-4 and GPT-4o with their advanced language understanding, enable learning from data, recognising patterns, predicting outcomes and finally improving through trial and error. However, despite their efficiency, these AI models are not infallible. Some seemingly harmless outputs can spread toxic misinformation or cause harm in critical areas like healthcare or legal advice. These instances underscore the dangers of blindly trusting AI-generated content and highlight the importance and the need to understand its limitations.

Defining the Problem: What Constitutes “Nonsensical Answers”?

Harmless errors due to AI nonsensical responses can be in the form of a wrong answer for a trivia question, whereas, critical failures could be as damaging as wrong legal advice. 

AI algorithms sometimes produce outputs that are not based on training data, are incorrectly decoded by the transformer or do not follow any identifiable pattern. This response is known as a Nonsensical Answer and the situation is known as an “AI Hallucination”. It can be factual inaccuracies, irrelevant information or even contextually inappropriate responses. 

A significant source of hallucination in machine learning algorithms is the bias in input that it receives. If the inputs for the AI model are full of biased datasets or unrepresentative data, it may lead to the model hallucinating and producing results that reflect these biases. These models are also vulnerable to adversarial attacks, wherein bad actors manipulate the output of an AI model by tweaking the input data ina subtle manner.

The Need for Policy Intervention

Nonsensical AI responses risk eroding user trust and causing harm, highlighting the need for accountability despite AI’s opaque and probabilistic nature. Different jurisdictions address these challenges in varied ways. The EU’s AI Act enforces stringent reliability standards with a risk-based and transparent approach. The U.S. emphasises creating ethical guidelines and industry-driven standards. India’s DPDP Act indirectly tackles AI safety through data protection, focusing on the principles of accountability and consent. While the EU prioritises compliance, the U.S. and India balance innovation with safeguards. This reflects on the diverse approaches that nations have to AI regulation.

Where Do We Draw the Line?

The critical question is whether AI policies should demand perfection or accept a reasonable margin for error. Striving for flawless AI responses may be impractical, but a well-defined framework can balance innovation and accountability. Adopting these simple measures can lead to the creation of an ecosystem where AI develops responsibly while minimising the societal risks it can pose. Key measures to achieve this include:

• Ensure that users are informed about AI and its capabilities and limitations. Transparent communication is the key to this.  
• Implement regular audits and rigorous quality checks to maintain high standards. This will in turn prevent any form of lapses.
• Establishing robust liability mechanisms to address any harms caused by AI-generated material which is in the form of misinformation. This fosters trust and accountability.

CyberPeace Key Takeaways: Balancing Innovation with Responsibility

The rapid growth in AI development offers immense opportunities but this must be done responsibly. Overregulation of AI can stifle innovation, on the other hand, being lax could lead to unintended societal harm or disruptions. 

Maintaining a balanced approach to development is essential. Collaboration between stakeholders such as governments, academia, and the private sector is important. They can ensure the establishment of guidelines, promote transparency, and create liability mechanisms. Regular audits and promoting user education can build trust in AI systems. Furthermore, policymakers need to prioritise user safety and trust without hindering creativity while making regulatory policies. 

We can create a future that is AI-development-driven and benefits us all by fostering ethical AI development and enabling innovation. Striking this balance will ensure AI remains a tool for progress, underpinned by safety, reliability, and human values.

References

• https://timesofindia.indiatimes.com/technology/tech-news/googles-ai-chatbot-tells-student-you-are-not-needed-please-die/articleshow/115343886.cms
• https://www.forbes.com/advisor/business/ai-statistics/#2 
• https://www.reuters.com/legal/legalindustry/artificial-intelligence-trade-secrets-2023-12-11/ ‍
• https://www.indiatoday.in/technology/news/story/chatgpt-has-gone-mad-today-openai-says-it-is-investigating-reports-of-unexpected-responses-2505070-2024-02-21

Executive Summary:

This report deals with a recent cyberthreat that took the form of a fake message carrying a title of India Post which is one of the country’s top postal services. The scam alerts recipients to the failure of a delivery due to incomplete address information and requests that they click on a link (http://iydc[.]in/u/5c0c5939f) to confirm their address. Privacy of the victims is compromised as they are led through a deceitful process, thereby putting their data at risk and compromising their security. It is highly recommended that users exercise caution and should not click on suspicious hyperlinks or messages.

False Claim:

The fraudsters send an SMS stating the status of delivery of an India Mail package which could not be delivered due to incomplete address information. They provide a deadline of 12 hours for recipients to confirm their address by clicking on the given link (http://iydc[.]in/u/5c0c5939f). This misleading message seeks to fool people into disclosing personal information or compromising the security of their device. 

‍

‍

The Deceptive Journey:

• First Contact: The SMS is sent and is claimed to be from India Post, informs users that due to incomplete address information the package could not be delivered.

• Recipients are then expected to take action by clicking on the given link (http://iydc[.]in/u/5c0c5939f) to update the address. The message creates a panic within the recipient as they have only 12 hours to confirm their address on the suspicious link.

• Click the Link: Inquiring or worried recipients click on the link.

• User Data: When the link is clicked, it is suspected to  launch possible remote scripts in the background and collect personal information from users.

• Device Compromise: Occasionally, the website might also try to infect the device with malware or take advantage of security flaws.

The Analysis:

• Phishing Technique: The scam allures its victims with a phishing technique and poses itself as the India Post Team, telling the recipients to click on a suspicious link to confirm the address as the delivery package can’t be delivered due to incomplete address. 
• Fake Website Creation: Victims are redirected to a fraudulent website when they click on the link (http://iydc[.]in/u/5c0c5939f) to update their address.
• Background Scripts: Scripts performing malicious operations such as stealing the visitor information, distributing viruses are suspected to be running in the background. This script can make use of any vulnerability in the device/browser of the user to extract more info or harm the system security.
• Risk of Data Theft: This type of fraud has the potential to steal the data involved because it lures the victims into giving their personal details by creating fake urgency. The threat actors  can use it for various illegal purposes such as financial fraud, identity theft and other criminal purposes in future.
• Domain Analysis: The iydc.in domain was registered on the 5th of April, 2024, just a short time ago. Most of the fraud domains that are put up quickly and utilized in criminal activities are usually registered in a short time.
• Registrar: GoDaddy.com, LLC, a reputable registrar, through which  the domain is registered. 
• DNS: Chase.ns.cloudflare.com and delilah.ns.cloudflare.com are the name servers used by Cloudflare to manage domain name resolution. 
• Registrant: Apart from the fact that it is in Thailand, not much is known about the registrant probably because of using the privacy reduction plugins. 

• Domain Name: iydc.in
• Registry Domain ID: DB3669B210FB24236BF5CF33E4FEA57E9-IN
• Registrar URL: www.godaddy.com
• Registrar: GoDaddy.com, LLC
• Registrar IANA ID: 146
• Updated Date: 2024-04-10T02:37:06Z
• Creation Date: 2024-04-05T02:37:05Z (Registered in very recent time)
• Registry Expiry Date: 2025-04-05T02:37:05Z
• Registrant State/Province: errww
• Registrant Country: TH (Thailand)
• Name Server: delilah.ns.cloudflare.com
• Name Server: chase.ns.cloudflare.com

Note: Cybercriminals used Cloudflare technology to mask the actual IP address of the fraudulent website.

CyberPeace Advisory:

• Do not open the messages received from social platforms in which you think that such messages are suspicious or unsolicited. In the beginning, your own discretion can become your best weapon.
• Falling prey to such scams could compromise your entire system, potentially granting unauthorized access to your microphone, camera, text messages, contacts, pictures, videos, banking applications, and more. Keep your cyber world safe against any attacks.
• Never reveal sensitive data such as your login credentials and banking details to entities where you haven't validated as reliable ones.
• Before sharing any content or clicking on links within messages, always verify the legitimacy of the source. Protect not only yourself but also those in your digital circle.
• Verify the authenticity of alluring offers before taking any action.
  

Conclusion:

The India Post delivery scam is an example of fraudulent activity that uses the name of trusted postal services to trick people. The campaign is initiated by using deceptive texts and fake websites that will trick the recipients into giving out their personal information which can later be used for identity theft, financial losses or device security compromise. Technical analysis shows the sophisticated tactics used by fraudsters through various techniques such as phishing, data harvesting scripts and the creation of fraudulent domains with less registration history etc.  While encountering such messages, it's important to verify their authenticity from official sources and take proactive measures to protect both your personal information and devices from cyber threats. People can reduce the risk of falling for online scams by staying informed and following cybersecurity best practices.

‍