The challenge: Managing the threat of malicious content at scale
When used appropriately, social media can be a force for good: it encourages communication and collaboration, making it easy for us to share and discuss ideas and issues. But sadly, social media is also prone to misuse. In the wrong hands, it can be used as a vehicle for misinformation, quickly spreading false facts or news to a large audience. In the worst cases, it provides a platform to promote extreme views, violence, and terrorism.
This threat posed a critical challenge for a leading political and economic commission. With our track record in identifying and moderating misinformation at scale and at speed, Firstsource was asked to:
- Measure and analyze the threat of violent extremism and terrorism across leading social media platforms
- Provide moderated data to train the customer’s AI/ML model to automatically detect rogue content in real-time
The goal was to suppress harmful content and protect free speech.
The solution: Talented trust and safety experts and technology working together to mitigate risk
Trust & Safety experts on our content moderation services team worked with the client and their partners to develop a moderation process for all content types across five social media platforms – Facebook, Twitter, Instagram, YouTube and TikTok. We developed workflows that focussed on identifying risk at scale while streamlining operations to maximize the efficiency and effectiveness of the customer’s budget.
Drawing on the native language and cultural nuance skills of our agents, we analyzed content in twelve different languages and curated a list of potentially malicious keywords, signs, and symbols. The data from this analysis was used to train ML-based classifiers and rules engines to detect any and all high-risk and potentially harmful content across five different platforms, in real-time, so that our skilled team of content moderators could step in and take appropriate action.
Rigorous quality tracking and evaluation measures were established to monitor and optimize our approach. The team also tracked channel behavior and generated new insights to inform and enhance the political and economic commission’s wider Trust & Safety initiatives to ensure a safer future for their citizens.
The results: Efficient and effective content moderation keeps citizens safe
Firstsource combined the very best of automated content moderation technology with human experience and expertise to deliver outstanding results in record time:
- Over 1 Mn pieces of content were collected and effectively analyzed to remove any potentially harmful language and imagery
- We attained consistent quality scores of more than 98% for content search and validation. This gave the customer peace of mind that their platform, and their citizens, were in safe hands
- Our trusted approach and streamlined operations delivered a 25% reduction in operating costs for our client to invest in further initiatives to keep citizens informed and safe
- Insights generated from our social media behavioral analysis are being used to shape the customer’s Trust and Safety strategy for other platforms and initiatives
Our client hit all their goals in addressing their critical and complex security challenges at scale.
