The Evolution of Trust & Safety in 2024
The trust and safety landscape has emerged as a critical priority for countries and organizations worldwide, far beyond its traditional role of content moderation. In 2024, this space evolved rapidly driven by technological advancements and significant global events. GenAI led innovations have accelerated the rise of new and complex online harms, requiring trust and safety professionals to adapt continuously. Simultaneously, sociopolitical developments, including elections in 64 major countries, and complex geopolitical tensions have further put the trust and safety practice under a spotlight.
Today, trust and safety is no longer confined to content moderation or reactive policies. It is now a critical component of platform strategy, brand reputation, and user loyalty. As we analyze the key trends of 2024, three primary themes emerge:
1. Increasing adoption of AI and automation
AI-driven solutions facilitate large scale content moderation, detection of harmful behaviors, and automation of responses to low-severity incidents. These solutions enable efficient resolution by enabling human moderators to focus on nuanced, context-heavy cases.
However, they are facing challenges such as algorithmic bias and misclassifications. Even advanced AI models are exhibiting certain biases and inaccuracies, particularly struggling with cultural context leading to misclassifications. It is critical to maintain human oversight to ensure fairness and accountability.
2. Changing regulatory environment
The regulatory landscape grew more complex in 2024 as governments worldwide intensified their focus on online safety, privacy, and accountability. Regulations such as European Digital Services Act and Global AI Regulations in the AI space have been introduced to promote ethical use of AI, particularly in sensitive areas like misinformation and user safety. These changes have compelled companies operating in the EU to invest in compliance by enforcing tight deadlines and heavy penalties for non-compliance.
3. Shift in user expectations and behaviors
Users and advocacy organizations are demanding safety, transparency and fairness from platforms. This has led platforms to prioritize user wellbeing, integrate transparency in their safety policies, provide clearer reporting tools to reassure users. Proactive testing of malicious scenarios through red teaming and pen-testing is also becoming a critical element of safety as companies develop new protection strategies against misinformation and fraud.
Technological Innovations in Trust and Safety
This year saw significant advancements in Trust and Safety technology. Innovations driven by AI advancements spurred the adoption of GenAI, predictive analytics and wellness tools, offering significant improvements to enable platforms to enhance user protection.
- GenAI Tools: Generative AI emerged as a cornerstone of content moderation in 2024, offering platforms the ability to analyze and respond to user-generated content more efficiently. GenAI models enabled faster detection of harmful content, including text, images, and videos.Further improvements in the understanding of cultural nuances, reducing false positives and negatives in content moderation will enhance the role and adoption of GenAI tools.
- Predictive Analytics: In 2024, trust and safety experienced a paradigm shift from reactive to proactive approach. Predictive Analytics enabled practitioners to detect risk early, enhance prioritization, effectively moderate high-risk incidents, and respond more swiftly to harmful content.Opportunities for improvement here include addressing data privacy risks from heavy reliance on user data, and false alarms.
- Wellbeing and Mental health tools: AI-powered wellness tools provide personalized support by offering stress management and mindfulness solutions tailored to individual needs.
The wellness practice, however, requires investment and proactive work across all kinds of business. Companies need to further deepen their understanding of the effectiveness of wellness tools, especially in workplaces where not all variables of conscious exposure are adequately accounted for.
Building resilience in the Trust and Safety practice
2024 underscored the critical need for resilience within trust and safety teams. Hybrid and remote working models along with the ever-growing tide of harmful content and complex regulations has made team cohesion, morale and wellness paramount. The field of Trust and Safety has always been high-pressured, but 2024 underscored the critical need for resilience.
Implementing the right strategies such as regular check-ins, virtual collaboration tools and an intentional approach to wellness for remote workers made a difference in the implementation of trust and safety initiatives.
What’s next for Trust & Safety?
The lessons of 2024 provide a good view of what 2025 will look like. Platforms must move beyond simply managing challenges and lead the way in creating trust, safety and resilience for users and teams alike.
Stay tuned for Part 2 of this blog series, where we will dive into 2025 trends, including the rise of proactive risk management, user safety as a competitive advantage, and the establishment of AI-driven Trust & Safety systems.
Connect with our team to explore the strategies and areas of collaboration and enhance your trust and safety practice.
