Human Rights-Based Approaches to Preventing Terrorists and Violent Extremists from Exploiting Digital Platforms
In March 2024, the Global Internet Forum to Counter Terrorism (GIFCT) worked with BSR to convene an expert discussion on practical human rights-based approaches to preventing terrorists and violent extremists from exploiting digital platforms with current and incoming GIFCT member companies.
This briefing note has been developed by BSR in coordination with GIFCT to summarize the key questions addressed in the briefing. The speakers were from Article 19, BSR, GIFCT, and the U.N. Office of Counter Terrorism (UNOCT), however, the discussion was conducted under the Chatham House Rule, so the content is not attributed to specific speakers.
Summary
- GIFCT’s approach to human rights is based in large part on the United Nations Guiding Principles on Business and Human Rights (UNGPs). The first pillar of the UNGPs covers the “state duty to protect human rights” while the second covers “the corporate responsibility to respect human rights.” Both pillars are directly relevant to GIFCT’s mission as both governments and companies participate in GIFCT’s work.
- The UNGPs set out a four-step human rights due diligence process that all companies should follow (though smaller companies may be more informal in their approach): conducting assessments, taking appropriate action, tracking effectiveness, and communicating publicly.
- There are two main categories of human rights risk to be assessed in the field of counterterrorism and violent extremism: terrorist and violent extremist content itself being a source of risk (e.g., terrorists exploiting platforms), and the risk that measures taken to prevent terrorist and violent extremist content result in adverse human rights impacts (e.g., over-moderation).
- When implementing mitigation measures to address these human rights risks it is important for companies to appreciate that compliance with human rights standards is ongoing. While no company will ever be 100% compliant, a commitment to improve over time is important.
- Monitoring the mitigation measures is crucial to understand how well they are working, whether they can be improved, and how methods may need to change over time.
- Transparency is essential, such as having unambiguous terms of service and publishing detailed transparency reports. Sanctions for non-compliance with terms of service should be appropriate and accompanied by access to a remedy mechanism for appeals.
Introduction
While respect for human rights and counterterrorism can sometimes be framed as being in tension, they are more typically reinforcing and complementary objectives, as lessons learned from international counterterrorism efforts have indicated. Violations of human rights are known drivers for terrorist and violent extremist content, so embedding human rights into GIFCT’s work recognizes that security at the expense of human rights is not sustainable or credible, and countering terrorist and violent extremism online is itself a human rights objective.
GIFCT’s approach to human rights is based in large part on the United Nations Guiding Principles on Business and Human Rights (UNGPs) and reflects international frameworks that affirm that counterterrorism efforts must comply with international law and human rights obligations. The first pillar of the UNGPs covers the “state duty to protect human rights” while the second covers “the corporate responsibility to respect human rights.” Both pillars are directly relevant to GIFCT’s mission as both governments and companies participate in GIFCT’s work.
While working with digital platforms, three main considerations for integrating human rights-based approaches in counterterrorism work include:
1. What does having a human rights policy mean in practice for companies?
Principle 16 of the UNGPs states that a human rights policy or statement setting out a commitment to respecting human rights is a core expectation for companies. While this is a minimum expectation of large companies, there is a trend of investors expecting this commitment from smaller companies too. Consequently, when GIFCT launched its Membership Criteria, it included the criteria requiring a public commitment to respect human rights in accordance with the UNGPs.
In best practice examples used by companies today, this commitment is not limited to the operations and supply chain of a company but also includes its products and services. Principle 16 also includes an expectation that policy commitments are informed by relevant internal and external stakeholders, approved at the most senior level of a company, embedded in company policies and procedures, and made publicly available.
According to the UNGPs, all companies have a responsibility to respect human rights, but Principle 14 of the UNGPs provides flexibility for smaller companies to achieve this expectation differently, such as through more informal processes and management structures than larger companies. For example, a commitment to human rights could be articulated in a few sentences (such as part of Terms of Service or Community Guidelines) and anchored in international human rights standards.
It is important for companies to appreciate that compliance with human rights standards is an ongoing work-in-progress and no company will ever be 100% compliant; a commitment to improve over time is important.
For some tech companies, a commitment to human rights has been limited to considerations around freedom of expression and privacy. However, a commitment to other relevant human rights is important too, such as non-discrimination, security of person, and access to culture. Human rights are interconnected and respecting rights to freedom of expression and privacy can enable the realization of other rights, such as freedom of association, freedom of assembly, and democratic participation.
The UNOCT recently published a “Guide for Human-Rights Based Approach to Countering Use of New Technologies for Terrorist Purposes” that provides information on how states should address the misuse of technology by terrorist actors. While this guidance focuses on the obligations of states under human rights law, it can also be useful for companies given their interaction with governments when pursuing counterterrorism objectives.
2. What does human rights due diligence in the context of counterterrorism and violent extremism online mean in practice?
The UNGPs are clear that human rights due diligence should be ongoing rather than a one-off activity that is never reviewed or repeated. The counterterrorism and technology ecosystem is continually evolving, which means that the accompanying human rights risks change over time as well.
Content moderation is a key tool used by companies in the technology sector to reduce the ability of bad actors to exploit digital platforms, promote terrorism, or spread violent extremist content. Transparency is an essential starting point in this endeavor, such as having unambiguous terms of service (TOS) and publishing detailed transparency reports. Sanctions for non-compliance with TOS should be appropriate and accompanied by access to a remedy mechanism for appeals.
Key points include:
- Users need to know what content is allowed and what content is prohibited. This should be available in all languages, relevant to the context, and easily accessible. Where a company has a list of prohibited organizations, consideration should be given to making this list publicly available.
- The Manila Principles specify information that companies should include in transparency reports related to government requests and a companies’ own content moderation. This includes what content has been restricted, why that content has been restricted, and a summary of the measures companies have taken (e.g., altering visibility of content).
- Users should be informed when content is restricted. If legal restrictions make it impossible to inform users, then other steps should be taken such as providing information about the laws governing a company’s decision to restrict content.
- Relevant resources include “Letting the Sun Shine in: Transparency and Accountability in the Digital Age” (UNESCO), the “Content Moderation Handbook” (Article 19), “Pathways to Meaningful Transparency” (GIFCT) and “Transparency Reporting: Good Practices and Lessons from Global Assessment Frameworks” (GIFCT).
The UNGPs set out a four-step human rights due diligence process which emphasizes the ongoing and holistic nature of due diligence. These steps should be the same for all companies, though smaller companies may be more informal in their approach:
- Assess (UNGPs Principle 18): There are two main categories of risk to be assessed in the field of counter terrorism and violent extremism. The first is terrorist and violent extremist content itself being a source of risk (e.g., terrorists exploiting platforms) and the second is the risk that measures taken to prevent terrorist and violent extremist content will result in adverse human rights impacts (e.g., over moderation; privacy; non-discrimination). In practice, companies need to understand what users, groups, and organizations present risk, assess how these users are using the platform today, and explore what factors could increase this risk in the future.
- Take action (UNGPs Principle 19): Taking appropriate action includes having clear policies in place, using automated tools to scale enforcement, making sure human review is working well, and ensuring that effective user reporting systems and appeals channels are in place.
- Track effectiveness (UNGPs Principle 20): Monitoring the mitigation measures is essential to understand how well they are working, whether they can be improved, and how methods may need to change over time.
- Communicate publicly (UNGPs Principle 21): The UNGPs emphasize the importance of public communications at a high level but offer companies some flexibility in how they communicate publicly in practice. The UNGPs expect that public communications should be in a form, frequency, and format that reflects the human rights impacts of the company and is accessible for the intended audience. The UNGPs also expect companies to provide sufficient information so that external stakeholders can understand the approach taken and assess the company’s performance. It was noted that new regulations in Europe (specifically, the EU Corporate Sustainability Reporting Directive and accompanying European Sustainability Reporting Standards) will add a more structured and formal dimension to company disclosures in the future.
3. How should human rights be factored into the definitions of terrorism and violent extremism, and the use of designation lists?
A major challenge for all companies operating internationally is that there is no universally agreed definition of terrorism, even if several international agreements and treaties define terrorist acts. State (or even sector) definitions of terrorism are often not consistent with international human rights law and can be very different to each other.
However, the main components of terrorism have been crystallized under international law and there are working definitions that companies can use. The U.N. advises companies to adopt or strongly consider the description contained within U.N. Security Council Resolution 1566 (2004). In addition, the 2010 Report of the Special Rapporteur on the Promotion and Protection of Human Rights and Fundamental Freedoms While Countering Terrorism offers a model definition of terrorism and a model offense of incitement to terrorism, which U.N. Security Council Resolution 1624 (2005) encourages states to prohibit. Relying on these definitions gives companies a better starting point even though they are not legally binding.
GIFCT’s hash sharing database inclusion criteria and Definitions and Principles Framework site are useful resources that can help companies and stakeholders navigate the different global frameworks. This definitions site includes a comparison of 80-plus global definitions of terrorism and violent extremism and a discussion about the pros, cons, and risk mitigations to deploy when using designation lists.
Companies can develop their own designation lists, which may include, for example, individuals and groups associated with ISIS and Al-Qaida listed in the sanctions list associated with the U.N. Security Council Resolution 1267 (1999) (known as the “Consolidated List”). However, the use of designation lists by individual companies can raise issues of human rights and transparency if they are not compiled and maintained in a way that is foreseeable and transparent.
However, it is important to note that, in contrast with terrorism, definitions and ways to designate violent extremism—or simply “extremism”—can be more contentious and less well defined, if at all. A growing number of governments define violent extremism in an overbroad manner to include content, groups, and organizations, as part of their efforts to curtail rather than protect civil rights.
Defining terrorism or violent extremism creates potential risks to freedom of expression, enshrined in Article 19 of the Universal Declaration of Human Rights and Article 19 of the International Covenant of Civil and Political Rights. Freedom of expression is not an absolute right, but it can only be limited in specific circumstances such as the protection of national security, public order, or when it is necessary for the protection of the rights or reputations of others.
Context, tone, and nuance are all important when assessing terrorist and violent extremist content, such as whether speech is satire or news media reporting. With the emergence of generative AI, there are heightened concerns about its impacts on terrorism and counterterrorism, and about it overcorrecting and restricting freedom of expression; automated tools can struggle to understand the nuance in many languages and some governments misuse national security as a justification to crack down on political dissent or opposition.
To limit abuse, government demands for taking down content should be interpreted as narrowly as possible to limit overreach or overmoderation, and companies can seek advice from the Global Network Initiative, which provides guidance on how companies can handle government requests in a rights-respecting manner.
Conclusion
Efforts to prevent and counter terrorism and violent extremism will only be effective, sustainable, and credible if they comply with international human rights obligations and standards. The UNGPs provide a strong foundation for both large and small companies, including measures to enhance transparency, monitor and revise approaches where necessary, and strengthen communications and public information. Taking human rights-based approaches to preventing terrorists and violent extremists from exploiting digital platforms also requires industry-wide awareness and partnership, and GIFCT seeks to provide a variety of forums to pursue multi-stakeholder collaboration and achieve more together than individual organizations could accomplish alone.
In order to support current and incoming members to integrate the human rights elements of GIFCT’s membership, the joining criteria and guidance to companies have evolved to include, (1) working with companies to develop a clear and public commitment to human rights in accordance with the UNGPs, (2) developing processes to review related policies and commitments regularly, and (3) ensuring companies continue to have access to cross-sector experts and support to inform their approaches. GIFCT and BSR will continue to work together to provide tech companies with expertise and training opportunities to further integrate human rights considerations and due diligence into their counterterrorism and counter-extremism efforts.
BSR is a sustainable business network and consultancy focused on creating a world in which all people can thrive on a healthy planet. With offices in Asia, Europe, and North America, BSR provides its 300+ member companies with insight, advice, and collaborative initiatives to help them see a changing world more clearly, create long-term value, and scale impact. For more information, see BSR’s website.
