Data-Driven and User-Centered Content Moderation

Introduction

Online platforms apply moderation interventions (MIs) to mitigate misbehavior. Today, MIs are one-size-fits-all, meaning that each intervention is applied in the same way for all users. However, not all users are the same, as they have diverse demographics, ideologies, and personalities.

Limitations of Current Approaches

This naive approach to content moderation is platform-centered and neglects user differences. Moreover, content moderation resembles art more than science. The design of MIs is based on common sense and intuition, and progress is sought via trial-and-error rather than via a rigorous scientific process.

The inevitable consequence is that current MIs have variable effectiveness, are highly unreliable, and fall short of the moderation needs.

Goals of DEDUCE

The ambitious goal of DEDUCE is to initiate a paradigm shift in content moderation by building the theoretical and methodological foundations to move from intuition-driven approaches enforced via one-size-fits-all MIs to science-driven strategies grounded on personalized moderation interventions (PMIs).

1. We will develop causal methods and indicators to evaluate the effectiveness and fairness of current content moderation practices.
2. Then, we will study how user characteristics influence the outcomes of moderation.
3. Finally, we will leverage the acquired knowledge to design and evaluate PMIs, a first-of-its-kind endeavor.

Data-Driven Approach

Our data-driven approach will enable us to evaluate in advance the effects of many MIs (what-if analyses) and to plan ahead their application, rather than to assess and correct afterwards.

Impact and Benefits

The high-gain nature of DEDUCE is evident, as it will open new directions of research (e.g., the design of PMIs), while also providing major practical and social benefits.

Our results will yield groundbreaking advancements in the theory and practice of content moderation, and will be embodied in:

• Practical guidelines for moderators and policymakers
• An open-source proof-of-concept system to support both human and automated moderation.