<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Checkpoint - Medium]]></title>
        <description><![CDATA[Checkpoint is a publication of Checkstep Ltd., a startup providing content moderation at scale to online communities. We publish articles related to disinformation, fact-checking, online moderation, and free expression. - Medium]]></description>
        <link>https://medium.com/checkstep?source=rss----eddfb7847266---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Checkpoint - Medium</title>
            <link>https://medium.com/checkstep?source=rss----eddfb7847266---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 07:02:19 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/checkstep" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Expert’s Corner with Zara Ward from Revenge Porn Helpline & Stop NCII]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-zara-ward-from-revenge-porn-helpline-stop-ncii-fc20d2d023b0?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/fc20d2d023b0</guid>
            <category><![CDATA[online-safety]]></category>
            <category><![CDATA[revenge-porn]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Mon, 19 Dec 2022 16:09:00 GMT</pubDate>
            <atom:updated>2022-12-19T16:00:31.038Z</atom:updated>
            <content:encoded><![CDATA[<p>This month we had the pleasure of interviewing <a href="https://www.linkedin.com/in/zaraw-rph/">Zara Ward</a>, senior practitioner at the Revenge Porn Helpline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/1*Fxh6XgPm2hoyZB4-r_DmeA.jpeg" /></figure><p>The <a href="https://revengepornhelpline.org.uk/">Revenge Porn Helpline</a> (RPH) is a part of the South West Grid for Learning charity and is a practical support service for adult (18+) victims in the UK who have been experiencing intimate image abuse (IIA). These issues could range from; images being shared without consent, threats to share imagery, webcam blackmail or voyeurism. RPH offer information on the laws around IIA and supports clients in reporting content where needed. Taking the burden away from victims of reporting illegally shared content.</p><p>The <a href="https://reportharmfulcontent.com/?lang=en">Report Harmful Content </a>(RHC) service offers people who are 13+ a human voice when reporting content to many social media platforms. The Report Harmful Content service are a web-only support service that can escalate harmful content previously denied removal from platforms.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FDRNeGvVk_y4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DDRNeGvVk_y4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FDRNeGvVk_y4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/832c0539759d7add476b31182ae22e90/href">https://medium.com/media/832c0539759d7add476b31182ae22e90/href</a></iframe><h3>Transcript</h3><p><strong><em>S: Welcome. Zara, I thought it’d be great to kick off the session. We have a quick intro about you and then we can take a deep dive into some of the important work the Revenge Porn Helpline is doing in the UK. Does that sound good?</em></strong></p><p>Yes, that sounds perfect. Great. so I’m Zara Ward. I work on the Revenge Porn Helpline.</p><p>I’ve been here about three years now, maybe more. I started off as a practitioner and I’ve worked my way up to senior practitioner. The main bulk of what I do is data research and technological research. So, yeah, I process all the data, do the reports, and showcase how technology could hinder people who’ve experienced IIA (Intimate Image Abuse) or support people, positively to protect their images and stuff like that.</p><p>My background is working in psychology, psychological research. So yeah, data is my thing!</p><p><strong><em>S: What was the reason that you joined the Revenge Porn Helpline?</em></strong></p><p>I just finished university and I kind of went off travelling for a year and I was like, I really, really want to get back into helping people and giving back and I saw an event come up for the Revenge Porn Helpline and I was like, hmm what is this? and so I looked into it and, and realised the scale of the problem and was like, “yeah, okay, this is what I can see myself doing” and here I am still doing it.</p><p><strong><em>S: Why was the Revenge Porn Helpline setup? What is its origin story?</em></strong></p><p>The Revenge Porn Helpline was set up in 2015. So, our charity sits underneath a wider organisation called the <a href="https://swgfl.org.uk/">South West Grid for Learning</a> and one of their helpline branches is the <a href="https://saferinternet.org.uk/professionals-online-safety-helpline">Professionals Online Safety Helpline</a>.</p><p>So that supports the Children’s Workforce, supports teachers who are experiencing issues relating to schools or relating to children. One of the managers, Laura, used to see quite a lot that intimate image abuse or revenge porn was quite commonplace in these kinds of situations, and it was actually against the law.</p><p>So as it became more of a problem, from 2015 when the law was introduced, they thought, “oh, well, let’s set up a helpline and see how popular it becomes” and over the years it’s increased in need and the support has become so much more developed and complex. So yes, we are very very busy!</p><p><strong><em>S: What is the current law in the UK with regards to intimate image abuse? And how is that being monitored and more importantly, enforced?</em></strong></p><p>That’s a really interesting one. So we know that the law was made to, well, it is illegal to disclose private sexual, intimate imagery with the intent to cause distress from 2015.</p><p>That is the specific law. It has to be, private sexual imagery, and it has to be with the intent to cause distress. So, as you can imagine, the intent to cause distress is a really difficult thing to enforce.</p><p>So we call it like the LOL clause. You can just say that you did it for a laugh or you did it because she was fit, or they were fit, or whomever and that gets you out of that causing distress tenant of the law. So that happened in 2015 and since then we’ve been kind of campaigning to get the law changed.</p><p>Recently the Law Commission has done a review and they recommend that the law can be adjusted to remove that attention from the base offence and have bolt on offences. Hopefully, depending on the status of our government for the next couple of years…we’re hoping that that will get enforced.</p><p>More recently in 2019 upskirting was made illegal, which was a part of the voyeurism offence. So that’s where somebody takes a picture up somebody’s skirt or or down their blouse.</p><p>Then even more recently again, threats to share intimate content was added to the Domestic Abuse Bill to make that its own specific offence but again, enforcing something like this is incredibly difficult because it’s online most of the time. You don’t have direct cause and effect, you don’t have evidence, you don’t have information. There is a lot of the ownership on the victim in order to keep a lot of this as logs and a lot of this is evidence.</p><p>Because these crimes are relatively new comparatively to other crimes, like, you know, murder is a very old crime, whereas intimate image abuse is under seven years, a lot of police forces aren’t knowledgeable about it. A lot of police forces feel very much like a fish out of water when it comes to enforcing these kinds of crimes. So we hear a lot of the time that it’s about a 50% split whether people get a positive or a negative response from the police and a positive response is just the police doing their job. We know that the institution of these kinds of crimes needs to change. I think it is slowly changing, but it does take a very long time to kind of drip down.</p><p><strong><em>S: I think there’s also the element, a large part, which is on the education piece. So everyone from the victim, to the police, all the way through to the person who’s sharing those images, they need to know what that means and the consequences that lead on from there. And I suppose again, even from the side of law enforcement, knowing what they need to do and where improvements can be made as well?</em></strong></p><p>We hear a lot of the times that still now police officers will say, “just turn your phone off” and “just forget about it. It’s online”. “It shouldn’t affect you or you shouldn’t take the picture in the first place or crop out your face, or your face isn’t in it. So why does it matter?”</p><p>Like there is such a different view of a physical body when it’s in a picture and it’s online than like a person’s body. There needs to be a lot of education around respecting people’s intimate content… like gifts. You wouldn’t just treat that horrendously. Hopefully the more we educate people on these subjects, and I think we are bringing about quite a woke generation in the next few years, which is brilliant.</p><p>I think people are learning a lot more about the importance of treating each other better and hopefully that will reduce the crimes, but who knows.</p><p><strong><em>S: In terms of the demographics, like gender, sexuality, ethnicity, age, have you identified from your research like any higher risk groups when it comes to revenge porn?</em></strong></p><p>So we don’t really capture that much demographic data because we are a confidential helpline but we know that when it comes to specific cultural groups, if somebody’s got a sexual image of image of them, that can be hugely different. They can suffer abuse, they can suffer real threat of violence, real threat on their lives but also somebody in a domestically violent or abusive relationship, any kind of picture of them getting out there can be absolutely dire for them, so it kind of depends on the circumstances.</p><p>It also depends on your job role. So if you are a teacher or a doctor etc, that could completely ruin your career.</p><p>Because the victim is still kind of slut shamed in these situations and all of this kind of stuff has to be investigated, we’ve had issues where people who are foster carers have been put under investigation because an image of them has been illegally shared and that is horrendous to not only attack a victim when they’re the most vulnerable, but to also take away so much of their personality.</p><p>We know that it disproportionately affects women having intimate images shared and women, when it comes to having intimate images shared, women’s images are shared more widely. So I think our statistic is like, overall, I think it’s like one image for a man and like 40 images for a woman, so per woman and per man who comes to the helpline with images to report. Just tiny. It’s a tiny amount. Like, like for male. I’m not saying that any image is too many images, but if it’s like so much more for women, that shows how much it’s being re-shared and how much it’s getting out there. Therefore its impacts are a lot more different.</p><p><strong><em>S: I know when we discussed before, we were speaking about the </em></strong><a href="https://www.bbc.co.uk/news/uk-62564028"><strong><em>BBC News article</em></strong></a><strong><em> that had come out about the secret world of trading nudes and also the articles that had come out about Reddit as well, and how they’re tackling this from a big tech perspective.</em></strong></p><p><strong><em>There is an offshoot of revenge porn labelled collect collector culture. Is this something you’re finding more of and how does that differ from a few years ago? Because I know now there’s more of a financial element that also plays into this. It’s not just the revenge part of someone who’s maybe been wronged, or feels justified, but there’s also this financial element that’s come into the mix over the last few years.</em></strong></p><p>Yes. I think because people are becoming more knowledgeable about how to find content of themselves and how to find if they have been talked about online, so people are becoming more knowledgeable about doing searches for themselves.</p><p>These people are becoming more privy to that kind of information. So instead of writing Zara for instance, they’ll do Z4RA to make it harder to find and then if that doesn’t work, they’ll, they’ll monetise it. They’ll put it behind a paywall, because if you want to see that person’s images so badly and they don’t have an OnlyFans; they’re not an open and free person to have that kind of content, then they’ll try and get it other ways and collector culture has been a huge issue that we’ve seen for a while. Way back when like Hunter Moore, IsAnyoneUp?, when he shared all the pictures of people locally that is pretty much what’s happening now but it’s geographical so people will find forums based in Devon or based in Cornwall or based in wherever, and it will break down further and further into different towns and they’ll say, “have you got images of such and such?” and somebody will anonymously post it and then, if somebody’s got a whole collection of that imagery, they can use that to sell the content. It’s the way that people are passing and trading content like a card game, it’s ridiculous.</p><p>Now I think the internet is such a big place, but also when you drill it down to that kind of granular level, it’s tiny to know that people in your local area could be potentially searching for your sexual content is a whole level of privacy issues that people haven’t even thought about before.</p><p><strong><em>S: I think the first thing that comes to my mind is also security and safety sharing certain images, we saw this on Reddit. It then spiralled into something where this lady was being blackmailed and there were threats against sharing the information and the images with her parents and she came from a Muslim background, so it then became something very sinister. I think that the guy who was caught and actually confessed to it said he didn’t do it with intent to injure or harm, but inadvertently put her into a very unsafe place she could have come to harm or, you know, her parents could have disowned her, for instance.</em></strong></p><p>Yeah, and people don’t think about that kind of stuff like I see that a lot on these kind of Reddit and those forums of being like, “What’s her @? What’s her @? What’s her Instagram? What’s her Twitter?” and it’s like, that’s none of your business; they are not a public person, they are not somebody who you can go on and scrutinise horrendously.</p><p>I think last year I did this research. I was really interested in the way that people become knowledgeable about their content being shared online because I was thinking, putting myself into that situation, how would I want to know the worst news imaginable and I would want my mum to tell me, because I love my mum.</p><p>My mum knows how to talk to me so I was like, “okay, let’s do a bit of research and figure out archivally, how people get to know this information”. So I broke it down into four things. I broke it down into; 1) they found themselves, so they could have used Google, they could have used facial recognition to find themselves; 2) they could have been told by the perpetrator who actually uploaded the images or made threats to share the images; 3) they could have been cold messaged by somebody on social media; or 4) they could have been told by their friends’ and family, so all of those options are absolutely horrendous but the one that was so surprising to me, is being cold messaged was 25% of the cases that I looked into. So that is people just blind messaging you, they found your Instagram or they found your Twitter and they’ve got your naked images and they’ve said, “I’ve seen these of you online”. That is the worst possible outcome to me personally, because there is no safeguard there. You are completely bamboozled by that situation.</p><p>You’ve not prepared yourself for it. Your parents would prepare you, your friends and family would prepare you for that kind of conversation. The perpetrator could potentially prepare you (maybe negatively). If you were searching for yourself, you could potentially prepare yourself but, being thrown into that conversation when you’re just looking at like, funny cat videos is bonkers to me and the outcomes are so diverse and that to me is the scariest thing because you’ve lost your privacy. You’ve lost your privacy to your body, privacy to where you’re sharing your personal photos and just your life generally.</p><p>It’s horrendous.</p><p><strong><em>S: The word that came to my mind was feeling violated. If that were to happen to me and someone were to message me and be like, “hey, I’ve got these images of you”. I would feel very unsafe. I would feel like there’d been a violation on me as a person and I suppose the impact of that psychologically is not just that one moment in time or what rolls out from that interaction. How do you get over that and then trust and feel safe using the internet or sharing information about yourself or pictures of yourself ever again?</em></strong></p><p>Yes, totally and, how you see and how you interact with other people.</p><p>All of our relationships now are mainly online. You know, you meet people when you go out for dates, you meet them online, or it’s very rare that you meet offline. Initial contact with somebody we are doing this online. Imagine that every single aspect of your life has been online and then you got this message saying, “here’s your images”.</p><p>Then most of the time they’re on a burner account and most of the time they’re not going to tell you where those images are; they’re just going to send you an image or a link or something really obscure. You then have to do all the detective work (potentially) as well as processing the issue, as well as then trying to understand where all your personal information has come from. Have you posted anything that could release your location, where you work, where you go to school? It completely changes the game and then every single person you interact with online, from that moment onwards, you’re going to be like, “have they seen it?”, “were they the person who released that content?”, it’s a completely different ball game.</p><p><strong><em>S: Obviously at Checkstep we work with lots of online platforms and one of the things that we were most interested in when we first got in touch with you was what online platforms can do to reduce the chance of intimate images being used and being traded in this way? What is their responsibility legally, but also morally as well?</em></strong></p><p>I think a block first policy is always a really good step. So if you’re getting a report of a private sexual image from somebody, just block it. Like take it seriously because then if somebody appeals it and says, “oh no, actually that’s a picture of a famous porn actress”, absolutely fine.</p><p>There’s more than enough porn on the internet. I think it wouldn’t matter if something went down for a couple of hours, but it does say that we are putting victims first. Rather than going, “we are putting commercial gain first or views first”, or whatever it is, which happens on a lot of adult websites where they’re like, “yeah, but it’s going to get views”.</p><p>It’s like, we know that, but that’s the point; we don’t want somebody’s really intimate moment getting 300,000 views when they’ve not allowed it to be online. If you have a block first policy and you appeal it later, it usually works quite well.</p><p>Obviously there should be a lot more safeguard to not allow that kind of content to be shared initially but places are going to share adult content and I think that’s really challenging.</p><p>I think having hashing lists as well is a huge plus if content has been verified being NCII. Just popping it through a hash bank and checking it and making sure there is no fingerprints that match up and making sure that that content has been shared in an okay manner, or maybe verifying people who are looking to share adult content.</p><p>It’s not perfect because you could still technically share illegally shared content if you are a verified creator. I think there’s a lot people can do to safeguard this sharing of adult content more generally because if you are not a verified person, then I think why should you be allowed to upload loads of sexual content? Then you’ve got no accountability to sharing that content in the first place. If you’re a production team, that’s fine, you’ve got accountability. If you’re a person, you should have accountability for what you’re uploading but unfortunately, that’s not the way the adult platforms work.</p><p><strong><em>S: The idea of block first is really interesting because we’ve seen with other types of online harms that actually, it could be online for a very small amount of time, but it only takes one person to copy that and then all of a sudden it can pop up everywhere.</em></strong></p><p><strong><em>I think that block first policies are something that’s definitely worth considering and thinking about from that side, it’s very difficult when there’s so much of this type of material being uploaded every millisecond. There’s things that can be put in place to help at least reduce the amount of time that it is visible online.</em></strong></p><p>Yeah, definitely. I think especially if you are on a platform that uploads a high amount of adult content, then it would be hard to say, “well, before you upload it, it can go into a moderation queue, get moderated, and then get uploaded”. That would just be creating a bottleneck but, having that saying of, “if you report something as a victim, we will take action on it straight away”, it reduces the need for human moderation because you only need to moderate it at the appeal stage, and also it shows again, to victims, that we are putting you first because there’s a lot of people who will not even think about reporting it because they’re like, “oh, if I report, they’re not going to do anything” but, if that’s at the forefront of your privacy notices, saying if you report something that you believe is intimate image abuse or IIA, we will take it seriously.</p><p><strong><em>S: And for the users themselves, obviously the platforms can put some safeguards in place, but what do you recommend that a user does if they discover that they’ve been a victim of revenge porn? Or even a family member or a friend of someone who finds out that someone has been a victim of revenge porn.</em></strong></p><p><strong><em>What would be your advice and what should they do next?</em></strong></p><p>It’s such a difficult situation, you don’t know how, you don’t know how much is going where. If you are a family member or a friend, I think getting in touch with a service, like an emotional support service, get some information from them about how you approach it, especially if that person’s got any additional needs, any mental health needs, just make sure that you’re safeguarding them as much as possible. Also, safeguarding yourself, not opening links., save that link if you can, or screenshot the chat and come to a website, not to toot our own horn, but like the Revenge Porn Hotline where it does have a step by step guide on how you can report content and what you could do so then you can be knowledgeable about that before you then go to your friend because you don’t want to go to your friend and say “oh, by the way, your images are online” and then not have any solution for them.</p><p>You can say, “right. I’ve spoken to these people because I just fully wanted to make sure that I was giving you the best advice. This is what they offer to do. Now, would you like me to help you through it? Would you like me to talk to them for you? Would you like to see the images?” Giving them that power and that ability to go, “actually, I’m not in the right headspace right now to view that content and to take that burden on. Yes, could you please talk to them for me?” or, “actually, no, I would like to take responsibility for making all those calls myself”. We do take calls from friends and from family members, because we appreciate the emotional load that that has on talking to somebody when you’re in that really hypersensitive state is really challenging.</p><p>If you have a link and it’s live, send it to us. We will do all the digging. We proactively search, we proactively report, we just get confirmation that they’re over 18, they’re the person in the image and that it has been obviously non-consensually shared and then we will go away like little squirrels and just keep going and keep going, keep going.</p><p>Our team probably reports on average about four or five hours a day. It’s all manual, it’s all done by a lovely team of pretty much three practitioners at this point. We are persistent and we are annoying. We will keep asking websites to take content down. We will find different ways to get websites to take content down. We offer people advice on how they can remove their personal information from Google, so if it’s attached to their name or, their sexual images are attached to images, so if you’ve got a professional profile, for instance, you don’t want your sexual images to be attached to that if somebody is going search for your name.</p><p>We offer advice on how to report it to the police, what the process looks like, how to get more legal support that’s for free. We have all of that information so, if you’re a friend or the person, we will give that information anonymously, we don’t need to know your information. We can just give you that little bit of advice. The more information you can give somebody, the better decisions that they can make for themselves going forward. Which I think is the best thing that they can have in that moment.</p><p><strong><em>S: I know we’ve spoken about it before where we were talking about the fact it doesn’t just end in the reporting and the removal of the image or the images that are online, but it’s the ongoing support that is often required to be able to sort of trust again and be able to interact online again. Where can people go for additional support and help if they need something more in terms of psychological support on these sorts of things?</em></strong></p><p>It’s really challenging because there’s no specific intimate image abuse support service. I think you’ve got Rape Crisis, which is more for offline for rape and sexual assault but, I think they do give some counselling. Obviously the NHS has counselling, but that’s more low level and it wouldn’t be for specific trauma. It’s a kind of piecemeal approach. In reality what we would love to do is to be able to offer some kind of emotional support, because we would see both sides; we would see the practical implications, but also the ongoing emotional implications and I would love to be able to do that or offer some pro bono counsellors in order to do that. That would be fabulous. Alot of people do say it’s like being sexually assaulted a million times. Like every time you open your phone, you are terrified that that person has found you and is showcasing your images again, and you open your phone probably a thousand times a day.; god knows how many times I open my phone. If that’s your house, for instance, how would you feel if somebody was just entering your house that many times a day and making you feel unsafe? That is not something that you can easily get over. People do need very complex trauma enforced therapy in these kinds of large scale cases.</p><p>We know that people have needed support and it has peaked and gone down depending on the way that their case is being managed at that time. They could have some time where no content is shared and then perhaps it could come back up again. It’s so case dependent, but ideally, yeah, we’d love to give it, I would just know that it’s going to be quite a long process but, to put that at the front of every single application you go for, for any kind of counselling is to say, “I’ve experienced this level of trauma” and then hopefully, they can adjust accordingly. I think that’s important.</p><p><strong><em>S: So in terms of our Q&amp;A today, that’s all the questions that I have. I want to say thank you Zara. Is there anything else you’d like to add or speak about today while we’re together?</em></strong></p><p>Uh, so I guess the only other thing is probably STOP NCII. So I know we’ve spoken a lot about the reaction of the stuff, so we have kind of noticed people share intimate images. It’s kind of obvious now. I don’t know anybody who hasn’t shared intimate images at this point.</p><p>We wanted to build something that was preventative rather than reactive. Last year we developed a stop non-consensual intimate image sharing tool. So Meta financed it and we got third party support from InCrev who have been amazing, the software developers.</p><p>They have built a hash bank, basically what already existed for child sexual abuse material, but for adult material. So what’s so different about this is that it’s on your device; the whole hashing algorithm would be on your device. It’s web-based and your images never leave your device.</p><p>They never come to me, or anybody else and the hash is the only thing that gets sent to the cloud; the hash of that intimate image. Then participating platforms can run the hash bank and match any content, then that would lead to a human moderator. It kind of gets you to the front of the queue when it comes to human moderation or when it comes to being notified.</p><p>It’s retrofitted as well. If your content has been shared previously, you add your image to the hash bank, or your hash to the hash bank later on down the line, it should retrofit and it should check previous uploaded content.</p><p>It seems to be really popular. It is genuinely really, really easy. So if you are feeling like there’s an image of yourself that is sexual and you don’t want it online or you want to safeguard it for now or in the future, I would really recommend just looking into it.</p><p>I think a lot of us here have used it. We’d be really bad practitioners if we haven’t and we see the use in it if you’re out dating or in the very normal sense if that’s what’s happening.</p><p>Also if there’s any kind of images of you that have been shared in the past and you have access to, so say you found an image of yourself on xHamster or wherever, you could download that image and hash it and then in the future, if we get xHamster onboard as a platform, it would take all that content off. It just offers a level of safeguarding.</p><p>Obviously you have to be an adult and it has to be a sexual image so genitals; it can be a video, you can have somebody else in the video, you can have as many people as you like, as long as you are in it; it doesn’t have to have your face in it, it just has to be naked or near naked.</p><p>I’m the first level of support so, if anybody has any questions, then they can just email us <a href="/service/mailto:stopncii@swgfl.org.uk">stopncii@swgfl.org.uk</a> and I will answer any questions. I’m the first port of call pretty much for any questions on that.</p><h3>Helpful links</h3><p><a href="https://revengepornhelpline.org.uk/">Revenge Porn Helpline</a></p><p><a href="https://swgfl.org.uk/">South West Grid for Learning</a> Charity</p><p><a href="https://stopncii.org/">Stop NCII</a> + explainer video <a href="https://revengepornhelpline.org.uk/information-and-advice/reporting-content/stop-ncii/">here</a></p><p><a href="https://reportharmfulcontent.com/?lang=en">Report Harmful Content</a> Service</p><p><a href="https://www.nhs.uk/nhs-services/mental-health-services/">NHS</a> — Access to mental health services</p><p><a href="https://rapecrisis.org.uk/">Rape Crisis</a></p><p><a href="https://www.victimsupport.org.uk/crime-info/types-crime/cyber-crime/image-based-sexual-abuse/#:~:text=You%20can%20also%20contact%20the,%2D4pm%2C%20Monday%20to%20Friday.">Victim Support</a> UK</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc20d2d023b0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-zara-ward-from-revenge-porn-helpline-stop-ncii-fc20d2d023b0">Expert’s Corner with Zara Ward from Revenge Porn Helpline &amp; Stop NCII</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Expert’s Corner with Marketplace Risk Founder Jeremy Gottschalk]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-marketplace-risk-founder-jeremy-gottschalk-8f14069a890d?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/8f14069a890d</guid>
            <category><![CDATA[marketplaces]]></category>
            <category><![CDATA[content-moderation]]></category>
            <category><![CDATA[online-safety]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Wed, 16 Nov 2022 09:02:53 GMT</pubDate>
            <atom:updated>2022-11-16T09:02:53.531Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.linkedin.com/in/jeremygottschalk/">Jeremy Gottschalk</a> is an expert in risk management and legal strategy for marketplace tech companies. With nearly 20 years of experience as a lawyer, operator and consultant to venture capitalists, founders and operators, he has become an industry-leading voice for legal strategy and risk management. Jeremy founded <a href="https://www.marketplacerisk.com/">Marketplace Risk </a>as an industry education, networking and knowledge-sharing platform for the marketplace ecosystem, and regularly consults venture capitalists, tech startups and vendors in this ecosystem. Jeremy holds a JD from Loyola and an MBA from Kellogg.</p><figure><img alt="Jeremy Gottschalk — CEO and Founder of Marketplace Risk" src="https://cdn-images-1.medium.com/max/384/1*ombKTeF623D_p1Lbd0cEqQ.jpeg" /><figcaption>Jeremy Gottschalk Founder and CEO of Marketplace Risk</figcaption></figure><p>Marketplace Risk is the most comprehensive source of risk management, trust &amp; safety, compliance and legal strategy education, networking and information sharing for marketplace startups to successfully launch, grow and exit. From the Blog, E-newsletter, Virtual Event Series, Masters Program, Platform Podcast and Community App, to the <a href="https://www.marketplacerisk.com/conference">Marketplace Risk Management Conference</a>, the <a href="https://www.marketplacerisk.com/global-summit-2022">Sharing Economy Global Summit </a>and beyond, Marketplace Risk is the first and only dedicated resource for startups to learn essential, proven information that enables them to avoid known pitfalls and distractions so they can focus on the growth and success of their venture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*8OQOGjuJRKM_iSy1PEC9lw.jpeg" /></figure><p><strong>1. Tell us a little about yourself? Your background and how you became founder of Marketplace Risk?</strong></p><p>I began practising law as an IP lawyer at two large, multinational firms out of law school. Early in my career, I was working with startups and, specifically, several early-stage marketplaces. These were the initial days of eBay and Craigslist, and well before Uber and Airbnb were founded. After practising for seven years, I joined a client in-house to build the legal function for the largest childcare marketplace at the time, Sittercity. I spent another seven years in-house, where I managed all aspects of legal, risk, and trust &amp; safety. Again, this was before “trust &amp; safety” was viewed as a priority, let alone a full-time profession. I recognized at the time that I was under-resourced, with big mandates. So, I organised a working group of childcare and childcare-adjacent marketplaces to learn and share information about risk management, trust &amp; safety, and legal strategy. Our group was created on the principle of helping one another and working towards improving the industry. Marketplace Risk was built on this foundation.</p><p><strong>2. Why was the Marketplace Risk setup? What is its mission?</strong></p><p>After seven years as in-house general counsel, I left to focus full-time on Marketplace Risk, and develop resources for startups to learn, network and share information about risk management, trust &amp; safety, compliance and legal strategy. By this point, it was evident that there was a lack of resources, and an urgent need for additional support to manage these issues in our fast-growing industry. In essence, there were no readily available and cost-effective resources for resource-constrained startup founders and executive operators. To this day, our mission is to provide complimentary and low-cost resources for marketplace and sharing economy startups to successfully launch, grow and exit. We do this through our global platform and community, which comes together for the Marketplace Risk Management Conference (San Francisco), the Sharing Economy Global Summit (London), various virtual events, and our curated digital content built by industry leaders.</p><p><strong>3. What does Trust and Safety (T&amp;S) mean for marketplaces?</strong></p><p>Trust &amp; safety means different things to different people across platforms and verticals. We know that trust is one of the biggest barriers to conversion and customer acquisition for marketplaces. Building trust is therefore the key to unlocking growth. But, you can’t build trust without safety at the core of your operations. Again, safety looks different for different platforms in different verticals, but fundamentally, safety means your customers (supply and demand) and your broader community are protected from harm. In addition, trust &amp; safety are a combination of human intervention and technology. Over time, trust &amp; safety will rely less on human intervention, but until then, high-calibre trust &amp; safety teams will use both.</p><p><strong>4. What do you see as the biggest operational challenges for T&amp;S in marketplaces?</strong></p><p>Trust &amp; safety are becoming more important to marketplaces, especially now that more prominent platforms publicly talk about trust &amp; safety, even releasing details about their own policies, practices and data. That’s fantastic for the industry. But, startups are typically resource-constrained and must balance investing in trust &amp; safety with developing technology, acquiring customers, etc. You don’t need trust &amp; safety if you have no customers. However, you’ll lose your customers if you don’t prioritise trust &amp; safety. As a result, it’s a resource prioritisation conflict that will always exist.</p><p><strong>5. Have you observed common types of harms across marketplaces?</strong></p><p>There is no common type of harm or fraud across all marketplaces. What I’ve seen is that bad actors invest time and energy to determine how to exploit different marketplaces in different ways. But, harm and fraud vary for goods marketplaces and services marketplaces. For example, goods marketplaces are likely to experience more financial and payment fraud than a services marketplace due to the nature of the transactions. On the other hand, services marketplaces will face a higher risk of physical harm to people, which is far more serious. What I tell founders and operators is to get out in front of bad actors — figure out how to exploit your marketplace before they do, so you can address those risks proactively.</p><p><strong>6. Have you seen any patterns or recurring themes across these marketplaces?</strong></p><p>With the explosion of marketplaces across every industry, bad actors and fraudsters are also increasing. And where there are similar marketplaces, bad actors employ the same or similar tactics until the vulnerabilities are addressed. But, it’s not always possible to mitigate all vulnerabilities, because marketplaces are the conduits or venues, they are not the ones perpetrating the bad acts, and they aren’t always involved in the transaction. So, it becomes nearly impossible to catch and eliminate all fraud, especially when you hit scale.</p><p><strong>7. What can platforms do to make their marketplace safer for both buyers and sellers?</strong></p><p>I think transparency is key — make your community aware of common scams and frauds — and provide as much information as possible to educate your community on precautions they can take to ensure smooth transactions. Certainly marketplaces can leverage identity verification solutions, background check products and various other screening tools. However, nothing is predictive of future bad acts. And, bad actors are getting more sophisticated, so there is no fool-proof tool, solution or method. But, in the aggregate, verification solutions, background check products and screening tools help improve the trust and safety of marketplaces.</p><p><strong>8. How do you feel about the proposed regulations for the Online Safety Bill in the UK and Digital Services Act in the EU, Section 230 in the US and Online Safety Act in Australia? Do you think these regulations will be helpful in improving marketplaces and gig economy platforms?</strong></p><p>Marketplaces have gone nearly unregulated for 25+ years despite explosive growth, which is astonishing. Unfortunately, during that time, we’ve witnessed that larger, well-known platforms don’t always act as good corporate citizens, which has brought scrutiny on all marketplaces, unfortunately. The result is the tendency of lawmakers to want to regulate. However, the very lawmakers proposing the regulations don’t understand the technology or the business model. So, the results are clumsy attempts to overregulate, which will threaten flourishing business models. Overall, the proposed regulations are well intentioned, but not necessarily well thought out or implemented.</p><p>In the US, marketplaces are protected by Section 230 of the Communications Decency Act, a federal statute that provides protection from legal liability for acts of marketplace users between each other. In other words, marketplaces are not legally responsible for the bad acts of their users. This protection is a huge asset to marketplaces, although there is much talk in Washington about scaling it back. And, two states — Florida and Texas — have even gone so far as to pass laws that limit the application of Section 230 in certain contexts. So, we expect the Supreme Court to review both state laws eventually, which could impair the broad protection of Section 230. And, that’s scary for anyone operating a marketplace in the US. Apart from that, the Supreme Court is currently reviewing <em>Gonzalez v. Google</em>, which will be the first time the highest court interprets Section 230. Again, the result of this case could spell disaster for certain applications of Section 230, if not having a broader, more far-reaching impact on the industry.</p><p><strong>9. How can platforms best prepare for these new regulations?</strong></p><p>The most efficient way is to visit <a href="https://www.marketplacerisk.com/">www.marketplacerisk.com</a> and attend the Marketplace Risk Management Conference (May 16–18 2023, San Francisco) or the Sharing Economy Global Summit (November 2023, London) — we are on top of all the regulations (and other high-priority issues) that impact marketplaces. We’ve also got a network of leading marketplace founders and experts dealing with these regulations head on and helping shape all aspects of the industry. So stakeholders in this space can access a myriad of resources to stay abreast of changes in the industry.</p><p><strong>10. What’s next for Marketplace Risk?</strong></p><p>We’ll continue to see incredible growth in all of our programs and events, which is how we provide the most value to our global community. We’ve been approached to launch additional conferences and summits in Asia, Canada and Europe, but we continue to focus on our core programs and events across the US and Europe. Apart from that, we’re evolving the platform to become an industry index, since we’re connected with so many marketplace stakeholders and have access to so much industry data and analytics. We’ll continue to expand our digital content to meet the demands of our growing global community.</p><h4><strong>Useful links</strong></h4><ul><li>Subscribe to the <a href="https://www.marketplacerisk.com/contact">Marketplace Risk newsletter</a></li><li>Join the Marketplace Risk Community (and get access to the app)</li><li>Attend the <a href="https://www.marketplacerisk.com/conference">Marketplace Risk Management Conference</a> and <a href="https://www.marketplacerisk.com/global-summit-2022">Sharing Economy Global Summit</a></li><li>Check out Marketplace Risk on-demand <a href="https://www.marketplacerisk.com/webinars">Virtual Events</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*8OQOGjuJRKM_iSy1PEC9lw.jpeg" /></figure><p><em>If you would like more information or are exploring options for AI enhanced moderation for your marketplace, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f14069a890d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-marketplace-risk-founder-jeremy-gottschalk-8f14069a890d">Expert’s Corner with Marketplace Risk Founder Jeremy Gottschalk</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Healthy Online Dating: Which swipe will keep you away from imposters?]]></title>
            <link>https://medium.com/checkstep/healthy-online-dating-which-swipe-will-keep-you-away-from-imposters-f10482cfa13e?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/f10482cfa13e</guid>
            <category><![CDATA[online-dating]]></category>
            <category><![CDATA[trust-and-safety]]></category>
            <category><![CDATA[content-moderation]]></category>
            <category><![CDATA[online-dating-app]]></category>
            <dc:creator><![CDATA[Vibha Nayak]]></dc:creator>
            <pubDate>Wed, 02 Nov 2022 07:01:58 GMT</pubDate>
            <atom:updated>2022-11-02T07:01:58.550Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QrZRC9dKylsms2Vf" /><figcaption>Online dating services seem to be gaining popularity in recent years, particularly due to the pandemic. However, while there are pros to using such matching making services, there are also cons (Unsplash)</figcaption></figure><p>In the quest to find a perfect match, people often fall prey to the wrong swipe. Thinking back, I ask myself of all those wrong swipes, what might it have led to, if my instinct didn’t urge me to run.</p><p>Very recently, I came across an <a href="https://www.wired.com/story/i-uncovered-an-army-of-fake-men-on-hinge/">article</a> by Lauren Goode on Vice, describing how she encountered an army of bots on Hinge. Not only could I relate, but it also made me think about my own experience on the app and the wider impact negative interactions can have on online daters in general.</p><p>Well, in keeping with the general MO of dating apps, I once connected with this super good looking lawyer, he had clever banter and didn’t hold back on compliments either. All a win-win for me’ why not right?</p><p>I wanted to come up with something original for our initial hang out, so I suggested axe throwing. The next thing I know I’m unmatched. My initial thought was maybe too violent of an activity for a first hang out? Oh well. But then he pops up on my Tinder as well. Guess he accidentally unmatched me on Hinge? Queen of second chances, I was like okay, let’s give this a go again. His Tinder profile was pretty much similar to his Hinge one, except for one additional picture. And this time around I didn’t suggest Axe Throwing 😅</p><p>But after asking him why he’d unmatched earlier, I was unmatched again, this time on Tinder. Guess he was my “Paulbot” as Lauren mentioned in her article, or just bad luck on my side. I won’t bore you with all the details of romance scams, as we already covered them earlier this year, all thanks to the <a href="https://medium.com/checkstep/the-tinder-swindler-e2f80831748c">Tinder Swindler</a>.</p><p>Nevertheless, this behaviour seems to keep repeating itself. It was termed “romance scams” / “swindling” in February. Fast forward a month, and we now have “<a href="https://theconversation.com/organized-crime-has-infiltrated-online-dating-with-sophisticated-pig-butchering-scams-177445">pig-butchering scams</a>”, which is practically the same as romance scams, i.e. they gain the victim’s trust over time and then con them in financial transactions. The only difference being that “pig-butchering” scammers are usually based in China.</p><p>Taking a step back, why do people partake in such behaviour? Could it be social acceptance? Is it because the social standards for a perfect match demand having a six-pack and being super wealthy? Or do these bad-faith actors just get their kicks out of scamming people?</p><p>Besides the financial loss, it also takes a toll on the victim’s mental health. When someone is truly out there, in the quest for a genuine love match, only to be dealt with by an imposter, that’s a huge setback. In addition to feeling depressed, this could also make the victim lose their self-esteem and feel even more lonely than they initially might be feeling.</p><p>“More than half a year on, Grace is still rebuilding her self-confidence.” ~ <a href="https://www.straitstimes.com/singapore/courts-crime/scam-victims-dont-just-get-hit-in-the-wallet-their-mental-health-also-suffers">Strait Times</a> (February, 2022).</p><p>The online dating services market is here to stay, as can be seen by its growing revenue of almost <a href="https://www.globaldatinginsights.com/news/online-dating-market-size-expected-to-reach-12-89-billion-by-2028/">USD $12.89 billion</a> expected by 2028. So what can we do to ensure that healthy interactions take place on these dating apps? Would better detection of profiles help? In many instances, there are also minors posing as adults on dating apps. While dating apps are already doing their best to keep their users safe, more needs to be done. A bad experience not only affects the user but can also cause churn for the dating app service.</p><p>Here are some tips on keeping safe:</p><ol><li>Always perform a reverse image search on profiles that are unverified. Better to be safe than sorry.</li><li>Look for inconsistencies in the user profile. You see any red flags, believe it and swipe left. Or if you’ve already matched, unmatch.</li><li>Try to match with verified profiles.</li><li>Don’t hesitate to report it if you have an uncomfortable experience with someone.</li><li>Ask for support if you’ve been impacted by abuse or loss as a result of an interaction on a dating app.</li></ol><p>If you would like more information or are exploring options for content moderation for your dating platforms, or you’d like to find out more about options for automatic AI enhanced moderation, contact us at <a href="/service/mailto:contact@checkstep.com">contact@checkstep.com</a>. Alternatively, you can also visit our website <a href="http://www.checkstep.com/">www.checkstep.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f10482cfa13e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/healthy-online-dating-which-swipe-will-keep-you-away-from-imposters-f10482cfa13e">Healthy Online Dating: Which swipe will keep you away from imposters?</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Checkstep Raises $1.8M Seed Funding to Combat Online Toxicity]]></title>
            <link>https://medium.com/checkstep/checkstep-raises-1-8m-seed-funding-to-combat-online-toxicity-51e1bf024d8b?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/51e1bf024d8b</guid>
            <category><![CDATA[online-safety]]></category>
            <category><![CDATA[content-moderation]]></category>
            <category><![CDATA[funding]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 15:37:11 GMT</pubDate>
            <atom:updated>2022-10-21T13:35:20.892Z</atom:updated>
            <content:encoded><![CDATA[<h4>Early stage startup gets funding for R&amp;D effort to develop advanced content moderation technology</h4><p>We’re thrilled to announce that Checkstep <a href="https://chk.st/pressrelease">recently closed a $1.8m seed funding</a> round to further develop our advanced AI product offering contextual content moderation. The round was carefully selected to be diverse, international, and with a significant added value to our business. <a href="https://www.crunchbase.com/organization/checkstep/company_financials/investors">Influential personalities of the tech industry</a> from the U.S. and U.K., but also from France, Bulgaria and Singapore are backing Checkstep, including many previous founders of online platforms, such as <a href="https://www.gov.uk/government/people/joe-white">Joe White</a> MBE, co-founder of Moonfruit.com and British Consul General in San Francisco, <a href="https://en.wikipedia.org/wiki/Jon_Oringer">Jon Oringer</a> who founded Shutterstock, a leading photography provider, as well as technology specialists, such as <a href="https://uk.linkedin.com/in/williamtp">William Tunstall-Pedoe</a>, who founded Evi, which became Amazon Alexa, and <a href="https://www.linkedin.com/in/sdianahu">Diana Hu</a>, who leads the AR platform at Niantic the company behind Pokemon Go.</p><p>This investment will go into R&amp;D to scale the software to deliver the additional functionality required to build a unique content moderation platform that manages the entire content lifecycle, from the policy definition to the management of the appeal process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*MoHrrmv52QbNyM7p2qadyQ@2x.png" /></figure><p>As the amount of misinformation, fake news and online abuse on our online platforms and forums continues to grow, major questions remain unanswered — both ethically and technically — over how platforms, users and content can best be moderated. Our CEO, <strong>Guillaume Bouchard, formerly Facebook’s AI Integrity Manager, </strong>sees 2021 as the year to bring about revolutionary changes for ensuring online safety. Prior to Checkstep, he exited his first business, Bloomsbury AI, to Facebook in 2018. To address the ethical implications inherent in using AI, we recently appointed Kyle Dent, ex-PARC, as our Head of AI Ethics. With Kyle’s focus on the intersection of people and technology, we aim to make sure that well-thought human decisions are at the core of the content moderation process and AI tools.</p><p>Checkstep was founded in 2020 with an initial focus on misinformation, but the founders soon realized that the mission of creating an inclusive internet involves addressing all potential forms of online harm from hate speech, child sexual abuse material (CSAM) to terrorist propaganda. Our company is building a 360-degree AI service set to help digital community providers prepare for the regulatory changes ahead while quickly scaling up their capacity to address and help growing online communities. Checkstep’s AI is also fully designed for interoperability through third party integrations, making Checkstep unique in its market segment.</p><p>We were featured in DCMS’s recent <a href="https://www.gov.uk/government/publications/safer-technology-safer-users-the-uk-as-a-world-leader-in-safety-tech">UK Safety Tech Sector Analysis</a> report as a new addition to their list of safety tech providers. The U.K. is a pivotal epicenter of opportunity across innovation and technology, and Checkstep was chosen through a rigorous application and selection process as just one of ten companies from over 200 applications this year into the Oxford Foundry accelerator’s fourth cohort, giving direct coaching advice from incredible advisors such as Kevin Roberts, Executive Chairman of Saatchi and Saatchi, a leading PR firm, and and resources.</p><p>Since its beginning, Checkstep’s leadership team has received direct coaching from the former CTO of Disney, Chairman and ex-CEO of Saatchi &amp; Saatchi, and the founder of Evernote, a leading productivity software company. This has enabled us to expand and confirm our business strategy, and prioritise our product roadmap much more effectively.</p><blockquote>“It is positive to discover startups like Checkstep that support platforms in taking content moderation to the heart of what they do, to build trust and safety with their users. What struck me about the Checkstep team is they also recognise the need to address AI bias, and with Guillaume’s pedigree in this sector, it seems like the right balance of technological know-how and ethical understanding and care, we are excited to see how Checkstep develops as a company and service provider over the coming months.” — Jon Oringer, Founder of Shutterstock</blockquote><p>We’re very proud to be backed by such strong investors who believe in our product.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51e1bf024d8b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/checkstep-raises-1-8m-seed-funding-to-combat-online-toxicity-51e1bf024d8b">Checkstep Raises $1.8M Seed Funding to Combat Online Toxicity</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Expert’s Corner with CEO of Recluse Laboratories Andrew Johnston]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-andrew-johnston-401c1851063e?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/401c1851063e</guid>
            <category><![CDATA[csam]]></category>
            <category><![CDATA[terrorism]]></category>
            <category><![CDATA[expert-corner]]></category>
            <category><![CDATA[content-moderation]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 15:36:15 GMT</pubDate>
            <atom:updated>2022-10-21T13:30:57.758Z</atom:updated>
            <content:encoded><![CDATA[<p>Our expert this month is ex-FBIer Andrew Johnston. Andrew is the CEO and co-founder of Recluse Laboratories. Drawing on twelve years of experience in cybersecurity, Andrew has worked with clients both in an incident response capacity as well as in proactive services. In addition to his private sector work, at the Federal Bureau of Investigation, Andrew served in the Cyber and Counterterrorism divisions where he performed field work and provided technical expertise to criminal and national security investigations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*yJdOAb7oQC7wdE27Ovmsrg.png" /><figcaption>Andrew Johnston is the CEO and Co-Founder of Recluse Laboratories.</figcaption></figure><p><strong>1. What was your motivation behind starting Recluse Labs?</strong></p><p>We started Recluse Labs to solve problems endemic to the threat intelligence industry. Threat intelligence, whether conducted in the private industry, academia, or government is reliant on a set of highly skilled individuals. These individuals are tasked with gaining access to adversarial communities, building believable identities, and extracting actionable information to form intelligence. Such individuals are few are far between, and the effect is that threat intelligence is often incredibly limited.</p><p>We’re passionate about cybersecurity and data science, and we believed that the combination of the two could enable us to have a far greater reach than organizations significantly larger than us. Since then, we’ve been working with industry peers, law enforcement, and intelligence experts to develop an automated, AI-enabled platform for collecting and analyzing intelligence.</p><p><strong>2. Are they specific patterns that online platforms should be mindful while tracking terrorist groups?</strong></p><p>One of the more interesting patterns is the mobility of many terrorist groups from one platform to another. In the past few years, there has been plenty of media coverage of up-and-coming social media platforms being swarmed with profiles promoting ISIL and other terrorist groups. Oftentimes, this creates significant challenges for a nascent platform, especially those that aim to have less restrictive moderation than some of the major players. It is worth noting that a strategy of aggressive banning doesn’t appear to be effective; terrorists have become used to profiles becoming disposable and regularly creating new accounts.</p><p>Consequently, the best approach to tracking and disrupting terrorist use of a platform has to occur at the content level. Naturally, simple solutions such as banning a set of words doesn’t scale well especially for a platform that caters to a multilingual audience. Likewise, human-centric approaches simply can’t scale to handle the volume and velocity of content that a healthy social media generates on a regular basis. Multilingual machine learning solutions are really the only answer to this problem that can both meet the scale and effectively identify novel terrorist content. We’ve dedicated a lot of research into developing terrorist content recognition systems that can meet the needs of social platforms, governments, and researchers.</p><p><strong>3. Quite recently, a known terrorist group took control of the Afghanistan government, i.e. the Taliban. What should the platform’s stance be on it?</strong></p><p>This is a hard question to navigate, as the answer will likely vary greatly depending on the platform’s philosophy. There is merit to the concept that the Taliban are a significant geopolitical force and denying their content as a policy hinders people from seeing the whole story. Moreover, hosting such content gives other users an opportunity to criticize, fact-check, and analyze the content, which could enable users who would otherwise be influenced by the propaganda to see the counterargument on the same screen.</p><p>Conversely, hosting such content means having to have very clear guidelines on when such content crosses the line to the point of being unacceptable. Left unchecked, those groups are known to publish disgusting or violent content that could alienate users and pose legal risks. Platforms then find themselves in the position of having to define what constitutes “terrorism” and merits removal. An improper definition could have side effects that impact benign marginalized groups.</p><p>In contrast, simply banning content promoting terror groups such as the Taliban keeps rules more clear, but doesn’t fully solve the problem. There are innumerable terror groups, and what precisely constitutes terrorism (and who is guilty of committing it) can be highly culturally specific.</p><p>Given that Recluse’s mission heavily involves building and deploying artificial intelligence to combat terrorism, we had to consider this question early on. We settled on targeting groups where there is a global consensus that they are terror groups. This definition ensures that we are never beholden to the political zeitgeist of any given country. Although this higher burden of consensus could inhibit us from targeting some groups we may personally find abhorrent, it ensures that we can operate without having to derive a “terrorist calculus” to evaluate every group.</p><p><strong>4. Child grooming and trafficking can be hard to track online. The New York Times did a piece on instances where Facebook often categorizes minors as adults when unable to determine their age. What are your thoughts on this?</strong></p><p>Identifying someone’s age is an incredibly difficult task — we all know someone who looks half their age. Consequently, even the best categorization system is going to have some level of error, regardless of how that system is implemented. That said, in the case of child exploitation, false positives and false negatives have significantly different impacts. Identifying potential child exploitation is paramount, whereas the penalty of a false negative is primarily additional workload for the moderation team. Of course, there is a balance to be had — a system with excessive errors is nearly as useless as having no system at all.</p><p>Thankfully, we don’t have to rely on our ability to identify minors as our sole tool against deplatforming child predators. Identifying and targeting language and behavior consistent with abusers can enable platforms to attack the problem at their source. In fusion with algorithms designed to identify minor users and child sexual abuse material, such techniques can better protect vulnerable groups even in the face of imprecise categorization.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=401c1851063e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-andrew-johnston-401c1851063e">Expert’s Corner with CEO of Recluse Laboratories Andrew Johnston</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Expert’s Corner with Policy Principal Andrew Bennett]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-andrew-bennett-2e8c534eab12?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/2e8c534eab12</guid>
            <category><![CDATA[expert-corner]]></category>
            <category><![CDATA[online-safety]]></category>
            <category><![CDATA[vc]]></category>
            <category><![CDATA[regulation]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 15:36:00 GMT</pubDate>
            <atom:updated>2022-10-21T13:29:00.961Z</atom:updated>
            <content:encoded><![CDATA[<h4>New online regulations are coming, and online platforms need to be prepared</h4><p><em>Andrew Bennett is a Policy Principal at </em><a href="http://formventures.vc/"><em>Form Ventures</em></a><em>, a U.K. VC fund investing in start-ups where regulation is a driver of success. Andrew’s expertise is in regulatory strategy, having worked across tech policy at the Tony Blair Institute and founded </em><a href="http://buttondown.email/txp"><em>TxP</em></a><em>, a network for people in tech and policy.</em></p><figure><img alt="Andrew Bennett, seated in mid discussion." src="https://cdn-images-1.medium.com/max/445/1*FOtuP4ui0AdpXMC4ZnDh6Q.jpeg" /><figcaption>Andrew Bennett is a Policy Principal at Form Ventures where he works on regulatory strategy and tech policy.</figcaption></figure><p><strong>How can online platforms keep up with upcoming regulations? What are the most imperative business implications? Do they really matter?</strong></p><p>The challenges for different online platforms vary. While some have no idea what their obligations will be, others have a sense of their duties but underestimate the time and effort required to build a fully-fledged trust and safety function. This is not just a new form to fill in or a tick-box exercise: new rules come with new enforcement powers, so many platforms will need to resource significant internal capability, systems and tooling to avoid challenges down the road. For companies trying to keep up, it’s worth looking out for regulators’ guidance. For example, despite the U.K. pausing the passage of its Online Safety Bill for now, the regulator, Ofcom, has published its <a href="https://www.notion.so/Checkstep-newsletter-interview-9f98e53031734afea53bc57f6d3a45ce">Roadmap to regulation</a> which sets out a timeline for actions services will need to take.</p><p><strong>What will be the consequences for platforms who aren’t compliant on day one?</strong></p><p>As Dorothy Chou said in<a href="https://formventures.substack.com/p/the-form-playbook-2-dorothy-chou"> our recent newsletter</a>, “the more start-ups have worked through the rules and institutions they want to operate within, the more freedom they will ultimately have to focus on the work they do best.”</p><p>That’s absolutely true of the online safety space, so despite the initial effort, it’s well worth getting on top of what will be required by cross-referencing the rules against how your service works, how you run your internal processes, and what’s in your community standards.</p><p>Fortunately, companies do still have time to get their heads around their obligations. In the EU, while large online platforms will need to comply within four months of being designated by the Commission, smaller platforms will have longer. In the U.K., once platforms’ duties come into force there will still be a period where Ofcom, the regulator,<strong><em> </em></strong>will be consulting on the language of its guidance and how it will enforce the rules on illegal harms, child safety and transparency reporting.<strong> </strong>Enforcement will also be proportional to the platform and risk in question. These will include fees for registering with the regulator. But we’re yet to see what that means in practice.</p><p>By 2024, however, services will need to be submitting their risk assessments and fully complying with the rules. If they don’t, akin to GDPR, they risk fines of up to £18m or 10% of global annual turnover (whichever is higher) — and under the EU’s DSA, fines could amount to 6% of global turnover. And complying with a full investigation from the regulator will also take focus and resources away from the business’ core job.</p><p><strong>What are the key differences between the EU’s Digital Services Act (DSA) and the U.K.’s Online Safety Bill (OSB)? What impact will they have on online platforms?</strong></p><p>Despite sharing a philosophy on how to regulate content, there are significant differences between the U.K. and EU’s approaches that mean platforms will need to prepare for different rules and different compliance processes — unfortunately this will mean additional costs and a limit to how much can be fixed and shared across different markets.</p><p>Specifically, the DSA goes further on rules around targeted advertising and external audit processes, while also having stronger protections for intermediary liability. For example, if Ofcom finds your service to be in breach of your duties, it may place a ‘proactive monitoring requirement’ on you. This was previously prohibited under the EU’s eCommerce Directive, except for terrorism and CSEA, and the DSA retains this approach. The OSB also bundles in other issues such as ID verification, duties for platforms to prevent children accessing pornography, and allowing the regulator to require platforms to use specific content moderation technology.</p><p><strong>Tighter and more stringent regulations can often be seen as governments abusing their power and restricting freedom of speech and expression. It has also been argued that the OSB will create potential obstacles for small and medium businesses. Do you think the proposed legislation sufficiently protects online innovation and free speech?</strong></p><p>There are clearly challenges with the OSB in its current form — from ambiguity in the wording of duties that might incentivize overzealous enforcement (undermining free expression), to contradictory duties that make it hard for services to comply at all. The new rules will also impose direct costs that make it harder not only for new social start-ups to scale, but also services in adjacent sectors — which policymakers may not even realize will be caught up in this — which have some user-to-user aspect to their work.</p><p>But it’s hard for anyone to argue that there isn’t a clear problem with online safety that needs addressing, and states have a sovereign right to legislate. For a small number of services, it might be worth engaging with the policy process to help improve the rules. But ultimately, there is still strong political momentum behind the OSB and the base case for all firms should be to adapt to the reality that this is happening.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e8c534eab12" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-andrew-bennett-2e8c534eab12">Expert’s Corner with Policy Principal Andrew Bennett</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Expert’s Corner with Lauren Tharp from Tech Coalition]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-tech-coalition-18719cac38e2?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/18719cac38e2</guid>
            <category><![CDATA[child-safety]]></category>
            <category><![CDATA[expert-corner]]></category>
            <category><![CDATA[online-safety]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 15:35:48 GMT</pubDate>
            <atom:updated>2022-10-21T13:28:29.698Z</atom:updated>
            <content:encoded><![CDATA[<p>For this month’s Expert’s Corner we had the pleasure of interviewing Lauren Tharp from the Tech Coalition.</p><p>The Tech Coalition is a global alliance of leading technology firms that have come together to combat online sexual abuse and exploitation of children. Because member companies have the same goals and face many of the same challenges, we know that collaborating to develop and scale solutions offers the most promising path towards a global solution to the problem.</p><p>Lauren Tharp is a Technical Program Manager based in Richmond, VA. She focuses her time helping companies adopt technologies to combat online child exploitation and abuse, and facilitates collaboration among a diverse set of industry members. Prior to joining the Tech Coalition, she worked as a Product Leader in the podcasting space where she learned the importance of Trust and Safety through the lens of brand safety. The answers included are not intended to represent any member of the TC or affiliated partners.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*sHwkBenI-hUx5c3t" /><figcaption>Lauren Tharp is the Technical Program Manager at Tech Coalition.</figcaption></figure><p><strong>1. What is the mission of the Tech Coalition? What is the idea behind it?</strong></p><p>The Tech Coalition facilitates the global tech industry’s fight against the online sexual abuse and exploitation of children. We are the place where tech companies all over the world come together on this important issue, recognizing that this is not an issue that can be tackled in isolation. We work relentlessly to coach, support, and inspire our industry Members to work together as a team to do their utmost to achieve this goal.</p><p>Every half second, a child makes their first click online — and the tools we all value most about the internet — our ability to create, share, learn and connect — are the same tools that are being exploited by those who seek to harm children. In this increasingly digital world, the technology industry bears a special responsibility to ensure that its platforms are not used to facilitate the sexual exploitation and abuse of children. Child protection is one place where our Members do not compete, but rather they work together to pool their collective knowledge, experience, and advances in technology to help one another keep children safe.</p><p>An example of how our work comes together is tech innovation — this is also the work I’m most passionate about. Our Tech Innovation working group exists first and foremost to increase our member’s technical capabilities to combat online CSAM. That means that even the smallest startups have access to the same knowledge and tools for how to detect and prevent CSAM as the largest tech companies in the world. We help members adopt existing technologies, such as technology to find known CSAM images and videos. We also fund pilots to innovate on new solutions, such as machine learning to detect novel CSAM to mitigate the need for human review. We also work closely with THORN — who have been invaluable partners in developing technology — and other subject matter experts throughout the industry to push innovation for our members.</p><p><strong>2. With more and more children spending time online, online child safety should always be a priority for social media platforms. Have you seen specific trends / patterns in terms of child abuse? Are they getting harder to detect?</strong></p><p>I’d say there are two major factors making it harder to readily detect online child sexual exploitation and abuse (OSCEA). The first is access. Many of us spend a significant amount of time online where we’re engaging, not only with trusted family and friends, but also have contact with strangers. This has largely been a success — think about cold outreach for a new job or finding peers who have a similar niche hobby. But the tradeoff to this access is that it also made it easier for bad actors to make contact with or groom children online. Recent <a href="https://www.thorn.org/blog/new-thorn-research-examines-youth-experiences-and-attitudes-about-online-grooming/">studies</a> have shown that nearly 40% of children have been approached by adults who they believe were trying to “befriend and manipulate them”. So I think we will continue to face the challenge of how to safeguard children online as bad actors subvert protective measures at an increasingly rapid pace.</p><p>The second factor is new content. Online CSAM has often taken the form of photos or pre-recorded videos, and so technology developed to detect those types of abuse. As users adopt new technologies such as live streaming, podcasting, direct message platforms, and gaming channels, the detection tools have to be trained towards those use cases. The difficulty is in keeping up with the pace of that change, and anticipating where abuse might occur next.</p><p><strong>3. When thinking about moderation, human moderation alone is unable to deal with the scale of the issue. What types of AI do you see innovating in this space to help companies keep up with increasing volumes?</strong></p><p>Human moderators are such an important part of keeping the Internet safe and free of child abuse imagery, but as noted, the scale of the problem requires innovative solutions (not to mention the psychological toll endured by many content moderators). To address these challenges, many companies use hashing technologies across photos and videos to detect and remove known CSAM. Hashing works by creating a digital “fingerprint” of photos or videos that have been deemed CSAM by a human moderator. These hashes are then stored in various databases that can be used across industry to automatically detect when the content is shared. The high degree of accuracy means less human moderation on content that we already know is violating.</p><p>While hashing is excellent at preventing the spread of known CSAM, it cannot detect new photos or videos. This is where true AI comes into play, also known as classifiers. Classifiers use machine learning to automatically detect if content falls into various categories, such as nudity, age, sexual acts, and more. By combining these categories, many companies can make quick decisions about which content should be escalated for review, versus content that does not meet the definition of CSAM.</p><p><strong>4. Organizations like NCMEC, Thorn and IWF help with detection of child sexual abusive material (CSAM) but what about child grooming? This is often harder to detect. How can platforms better prepare themselves?</strong></p><p>Grooming is a complex topic for many reasons, including the fact that there is no standard definition of what constitutes grooming, and it can vary by platform, language, culture, and other factors. In short, grooming is all about context. A classic example is the phrase “Do you want to meet on Friday?” which could be appropriate on a dating app, but is potentially inappropriate on a children’s gaming platform, again, depending on the context.</p><p>As a result, an organization’s approach to grooming detection must evolve over time to accommodate new trends. We typically recommend that companies start with keyword lists,</p><p>such as the <a href="https://safer.io/resources/csam-keyword-hub/">CSAM Keyword Hub</a>, which the Tech Coalition developed in partnership with THORN. The hub pulls known terms and slang related to online grooming and child sexual abuse so that organizations can begin to filter terms and adjust their content moderation strategies.</p><p>Increasingly we see a shift towards using AI to detect grooming by analyzing text (such as in public conversation channels) and by noticing behavioral signals (such as an adult user randomly befriending 100 minors in the span of an hour). The Coalition is working on training grooming classifiers on specific platform use cases, and continues to fund <a href="https://www.end-violence.org/articles/tech-coalition-safe-online-research-fund-awards">research</a> to understand perpetrators grooming strategies.</p><p><strong>5. Smaller platforms tend to have limited resources in dealing with online harms. Be it child safety or hateful content. What advice would you give them?</strong></p><p>My primary advice would be to reach out! The Tech Coalition has a robust set of resources, mentorship opportunities, innovative tooling, webinars, and much more to help the smallest companies get started. Additionally, many tech companies like Google and Meta offer free tooling and support to ensure platforms of any size can start mitigating abusive content from being shared on their platform. But if I could offer a true first step, it would be to start learning about the scale and nature of the problem as early as possible. If you allow content to be uploaded or conversations to occur, please consider child safety within your design and product flows to ensure children can use your platform without fear of harm.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18719cac38e2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-tech-coalition-18719cac38e2">Expert’s Corner with Lauren Tharp from Tech Coalition</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bants or Bullying: The impact of social media on minors]]></title>
            <link>https://medium.com/checkstep/bants-or-bullying-the-impact-of-social-media-on-minors-ebd552b18eb7?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/ebd552b18eb7</guid>
            <category><![CDATA[content-moderation]]></category>
            <category><![CDATA[cyberbully]]></category>
            <category><![CDATA[online-safety]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 15:35:12 GMT</pubDate>
            <atom:updated>2022-10-21T13:23:01.821Z</atom:updated>
            <content:encoded><![CDATA[<p>Author: <a href="https://www.linkedin.com/in/stephaniebornehughes/">Stephanie Borne</a>, Digital transformation and innovation strategist/ change maker and Inclusivity champion.</p><p>Background: Stephanie has worked for the UK’s most respected Organisations on Digital transformation and Digital communities, including ChildLine, The NSPCC, Plan International and Shelter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rSDgKm2rY9E29tr64n4Jkw.jpeg" /></figure><p><strong>With growing concerns over digital safety, and incidents (too often fatal) of exposure to harmful content, how can we make the internet a safer place for all?</strong></p><p>October was anti-bullying month in the U.S.; November 14th to the 18th will be Anti-Bullying Week in England and Wales. Clearly, the world is taking bullying and (increasingly) cyberbullying very seriously. Because Cyberbullying affects our children, the world is rallying around making digital safety a reality. But the internet has become a place where we have to watch our steps and be mindful of what we are sharing and who we may interact with, from online dating to gaming, or simply taking part in conversations in open platforms or closed forums. No one is really safe from harm.</p><p><strong>What follows is relevant whatever platform or community you are creating and managing, and a reminder that now is the time to take action.</strong></p><p>When my daughter turned 13, accessing her phone and checking in became trickier than before. Her new feeling of becoming a teenager and claiming her independence expressed itself in refusing to hand over her phone and claiming her right to privacy. And try to reason with a teenager without getting into an argument…</p><p>She would report some strange messages or behaviours that she identified as <em>bullyish</em>. But she asked me to please, please, please, not report it or contact the school, out of fear of embarrassment.</p><p>More worryingly, after finally seeing the messages, I struggled to understand what I was reading. Going through her contacts’ profiles was a painful exercise in sorting legitimate profiles of friends, using very entertaining aliases, from the predatory ones. And there were a few.</p><p>Even more worryingly, some messages from friends were akin to bullying, but she dismissed those as innocent banter. And I am a parent working in digital. I’ve designed and deployed social media campaigns, worked with all open platforms, I understand how it works.</p><p>In 2017, Molly Russell, 14 years old, took her own life after suffering a period of depression. The official report, available online states that “<em>The way that the platforms operated meant that Molly had access to images, video clips and text concerning or concerned with self-harm, suicide or that were otherwise negative or depressing in nature. The platform operated in such a way using algorithms as to result, in some circumstances, of binge periods of images, video clips and text some of which were selected and provided without Molly requesting them.”</em></p><p>Molly’s passing, although more related to harmful content than bullying, is still an extremely sad reminder that our children aren’t safe, that we struggle to keep them safe, that governments aren’t able to agree on regulations that are effective, and that platforms aren’t doing enough or not taking responsibility.</p><p>Andrew Walker, the coroner for the Northern District of Greater London, concluded Molly died from an act of self-harm while suffering depression and strongly highlighted the negative effects of online content.</p><p>He said the images of self-harm and suicide she viewed “shouldn’t have been available for a child to see”. Social media content contributed “more than minimally” to Molly’s death.</p><p><strong>However sad it is that a young girl had to lose her life to see things move in a more reassuring direction, things do seem to be moving…</strong></p><p>For example, The National Society for the Prevention of Cruelty to Children (NSPCC) in the U.K. is calling for tech platforms to take responsibility for protecting young people. Prince William says online safety for young people should be “a prerequisite, not an afterthought”.</p><p>In a first for both Meta, which owns Instagram, and Pinterest, senior executives had to give evidence under oath in a court in the U.K. Source <a href="https://www.bbc.co.uk/news/uk-england-london-63073489">BBC</a></p><p>And if governments do struggle with the “safeguarding versus freedom of speech” argument, regulations are about to get, if not clearer, but stricter. Like the 2016 EU General Data Protection Regulation (GDPR), that came about after the suicide of Olive Cooke, a 92 year old pensioner overwhelmed by the 3000 donations requests she would receive in a year, the new regulations may take a while to be implemented but they will come with the serious risk of fines that could be as high as 8 to 10% of a business’s turnover.</p><p><strong>All this to say that we can only hope for change.</strong></p><p><strong>The question is, who should and who will take preventive action?</strong></p><p><strong>So who’s responsible for helping tackle cyberbullying? Children?</strong></p><p>Children and teens are very resourceful. They know about bullying and try to help each other. As digital natives, there is very little difference to them between the real and virtual world. They adopt platforms and apps and navigate between one or the other seamlessly. They know there is the potential for harm, but can’t be expected to spot signs of abuse and are even less likely to be in control of online interactions.</p><p>Data around digital safety and cyberbullying is sparse or at best inconsistent. But we all know how damaging it can be. Not every child can talk to their parents; not every parent is digitally savvy enough to understand.</p><p>They are children, young people, and yes, need educating, but can’t be held responsible if the world they inherit is unsafe.</p><p><strong>Parents and educators?</strong></p><p>We all know too well how difficult it is to keep up with the digital world. We as adults give away our data and accept that platforms will serve us content that we haven’t asked for But most importantly, we know that reasoning with a teenager is not such an easy task.</p><p><strong>Simply put, parents, regulators and educators struggle to keep up.</strong></p><p>While platforms claim to deploy ways of keeping users safe, the onus seems to remain on users to understand how to protect themselves. Teachers are expected to educate young people in digital citizenship; parents are expected to do the same while at the same time monitoring their child’s activity.</p><p>Of course coining terms such as Digital citizenship<strong><em> “</em></strong><em>the ability to safely and responsibly access digital technologies, as well as being an active and respectful member of society, both online and offline”. </em>Source <a href="https://www.futurelearn.com/info/blog/what-is-digital-citizenship-teacher-guide">FutureLearn</a>, is helpful.</p><p>The term and concept is becoming more widely spread and shared.</p><p>But for educators, it comes on top of existing curriculums and means yet another topic to teach, understand and master in the first place. So there is still a long way to go.</p><p><strong>The platforms?</strong></p><p>If you search for what platforms are doing to prevent bullying or harmful content, they happily present a mix of human and tech measures. Yet, examples of distressing content flourish, with extremely serious and even fatal consequences on users’ mental health and lives.</p><p>When Adam Mosseri joined as Head of Instagram in 2019 he announced his intention to get serious about safety on the platform, “We are in a pivotal moment,” “We want to lead the industry in this fight.” Source: Time. New features were tested but we’re yet to see any real progress.</p><p>Facebook and Instagram’s own research showed that use of the platform made body issues worse for girls. Yet they will not accept the overall impact of their product on the mental health of the youth. Source / <a href="https://www.theguardian.com/technology/2021/sep/14/facebook-aware-instagram-harmful-effect-teenage-girls-leak-reveals">TheGuardian</a></p><p><strong>Systematically, platforms revert back to putting the responsibility on users to check privacy and safety features to protect themselves.</strong></p><p>The financial interest of platforms plays a big role in not pushing the safety agenda so much, or it encourages them to make compromises and focus on the financial gains. They are businesses after all and have targets to meet.</p><p>But this doesn’t excuse not putting everything in place to protect the communities they help create and facilitate. Gone are the days where one could expect communities to self-regulate. And when it comes to young people, and despite their amazing peer-to-peer support ethos, they should expect to be kept safe from harm, to enjoy and benefit from what these communities can offer.</p><p>So if we are managing an online community, the arguments that good moderation practices will help keep our brands safe from reputational harm, or from the risk of being fined or worse for not meeting regulatory requirements should already be motivation enough to review moderation tools and processes. But by facilitating engagement, the creation and sharing of content, we are ultimately responsible for our users and moderators safety and this should be a given.</p><p>In short, the internet will only be a safer place if everyone takes responsibility and acts upon harmful content and behaviours within our remit. Governments need to take action, parents need to educate and platforms need to be implementing the safest possible solutions. Apologising isn’t enough.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebd552b18eb7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/bants-or-bullying-the-impact-of-social-media-on-minors-ebd552b18eb7">Bants or Bullying: The impact of social media on minors</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Despite ban on “grooming” slur, Twitter takes no action on hateful posts]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/checkstep/despite-ban-on-grooming-slur-twitter-takes-no-action-on-hateful-posts-f994f4a7162a?source=rss----eddfb7847266---4"><img src="https://cdn-images-1.medium.com/max/1600/0*-oSWgkhyWIPsQJKt" width="1600"></a></p><p class="medium-feed-snippet">A new study shows a dramatic increase in anti-LGBTQ+ rhetoric on Twitter along with a &#x201C;child grooming&#x201D; narrative in paid Facebook ads&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/checkstep/despite-ban-on-grooming-slur-twitter-takes-no-action-on-hateful-posts-f994f4a7162a?source=rss----eddfb7847266---4">Continue reading on Checkpoint »</a></p></div>]]></description>
            <link>https://medium.com/checkstep/despite-ban-on-grooming-slur-twitter-takes-no-action-on-hateful-posts-f994f4a7162a?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/f994f4a7162a</guid>
            <category><![CDATA[florida]]></category>
            <category><![CDATA[lgbtq]]></category>
            <category><![CDATA[hate-speech]]></category>
            <category><![CDATA[twitter]]></category>
            <dc:creator><![CDATA[Kyle Dent]]></dc:creator>
            <pubDate>Tue, 16 Aug 2022 13:03:17 GMT</pubDate>
            <atom:updated>2024-06-05T18:50:46.033Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Expert’s Corner with Subject Matter Expert Carolina Christofoletti]]></title>
            <link>https://medium.com/checkstep/experts-corner-with-carolina-christofoletti-fa633fad7f58?source=rss----eddfb7847266---4</link>
            <guid isPermaLink="false">https://medium.com/p/fa633fad7f58</guid>
            <category><![CDATA[expert-corner]]></category>
            <category><![CDATA[child-sexual-abuse]]></category>
            <category><![CDATA[child-safety]]></category>
            <category><![CDATA[regulation]]></category>
            <dc:creator><![CDATA[Checkstep]]></dc:creator>
            <pubDate>Fri, 29 Jul 2022 15:04:58 GMT</pubDate>
            <atom:updated>2022-10-21T13:29:37.733Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Carolina Christofoletti is a Child Sexual Abuse Material (CSAM) subject matter expert, with vast experience both in CSAM Intelligence and CSAM Trust &amp; Safety. She currently leads the CSAM Blockchain Intelligence efforts at TRM Labs as part of the Threat Intelligence Team.</em></p><p><em>Her career includes an extensive CSAM academic background starting in 2017 at University of São Paulo — where she received an Alumni Excellence Award and a Law Degree. She holds an L.L.M on International Crime, Transitional Justice and Peace Procedures from Universidad Católica de Colombia (Colombia), a masters degree in Criminal Compliance from the Universidad Castilla La Mancha (Spain) and a masters degree in Cybercrime from the University of Antonio Nebrija (Spain) and is an L.L.M candidate for Digital Forensics at IPOG (Brazil).</em></p><p><em>Carolina Christofoletti is currently assigned as a CSAM researcher at the IGCAC Working Group, where she researches CSAM networks in depth and in partnership with national and international law enforcement agencies as well as at the Institute for Advanced Studies at the University of São Paulo (USP), where she researches CSAM Trust &amp; Safety. She also serves as a Consultant on CSAM Trust &amp; Safety matters.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*KdRqz_-MiTv4mlePqr4Brw.jpeg" /><figcaption>Carolina Christofoletti is a Child Sexual Abuse Material (CSAM) subject matter expert, with vast experience both in CSAM Intelligence and CSAM Trust &amp; Safety.</figcaption></figure><p>1. <strong>In a recent Linkedin post, you said “Image databases will not solve the CSAM problem.” Could you explain more about what you mean by that?</strong></p><p>CSAM Image Databases are bound by a logical contradiction.</p><p>Let’s suppose that:</p><ul><li>ID1 is a CSAM database (‘known’ is defined as existing in ID1).</li><li>Image A is a CSAM file that is part of ID1 (‘known’ CSAM images)</li><li>Image B and Image C are CSAM files that are not part of ID1 (‘unknown’ CSAM images)</li><li>Video A is a CSAM Video (CSAM media we know nothing about)</li></ul><p>(1) In order for a Image A to be added to ID1, it must first be found — and found in a manner that meets certain requirements, which is to say that Image A must be found (either directly or indirectly) by someone who has access to ID1, and who is willing to add it to the database. This someone is either a <em>Law Enforcement Agency</em> (Image A being thus part of a seizure) or a <em>CSAM Hotline</em> (Image A being often found by someone else and reported to the organization).</p><p>(2) It rapidly becomes quite clear that ID1 can only be useful if Image A is ever found and if the number of Image As does not exceed the scalability of ID1. And here lies the bottleneck:</p><p>The mere existence of ID1 has the harmful, unintended effect of obfuscating its main dependency, which is exactly a proactive approach for CSAM detection.</p><p>Instead of focusing on Image A (CSAM files flagged by ID1), we should be focusing on Image B and Image C (CSAM files that easily bypass it).</p><p>(3) Without constant updates, ID1 rotates in circles — as a CSAM image database serves (without further additions) only to identify known CSAM files (Image A), having no power over the unknown CSAM files (Image B and C).</p><p>Image A can only empower the discovery of Image B and C if:</p><ol><li>Image B and C appear before Image A and in the same digital environment, being still discoverable at the time Image A is detected. (linear condition), and</li><li>If Image A is used as a review flagger, rather than as a ‘tool’ for simply locating and taking down known CSAM files.</li></ol><p>(4) Because CSAM criminals do not report their sources and because the research around proactive detection is still poorly explored, Image B and C will hardly ever enter ID1.</p><p>Moreover, Image B and C might not be reported to or be found by a CSAM Hotline or a Law Enforcement Agency with access to ID1.</p><p>Even if Image B and C are actively circulating on the Internet, the fact that they circulate only among a closed groups of offenders who never report them, the fact that Image B and Image C are so severe that someone can fear their own prosecution by reporting it and the lack of a ‘proactive’ approach of CSAM hotlines to really ‘look for those files’ might, often, eliminate the chances of having Image B and C ever inserted into ID1.</p><p>(5) Even if Image A, Image B and Image C have the potential of being added to ID1, the same possibility isn’t available for Video A — since ID1s are image databases, which do not include CSAM videos.</p><p>A possible solution for this case would be extracting Image A (a CSAM scene) from Video A and enable the detection per similarity. Although the technology for this process already exists, it unfortunately remains poorly explored up to this point.</p><p>(6) Even though online platforms often expect Image A to be directly uploaded into their platforms (in order to trigger ID1), this hardly ever happens.</p><p>A quick observation into CSAM networks will show that the way Image A commonly enters online platforms is through an external link. The finding of Image A depends, as such, on the opening AND scanning of those links.</p><p>And here, the problem begins, as CSAM criminals tend to look for file hosting platforms with poor or low integration with ID1, and CSAM Trust &amp; Safety teams are not equipped with CSAM detection tools based on ID1 or that enable automatic detection in this case.</p><p>At this point, we realize how dependent on ‘opening the links’ and ‘human reviewers’ CSAM Trust &amp; Safety still is. A possible solution for this would be working on ‘risk scoring’ metrics for unknown links.</p><p><strong>2. The EU recently beefed up their regulations to ensure online child safety. How important do you think government regulations are or not to online child safety?</strong></p><p>The main risk of any regulation is getting the <em>“what to regulate”</em> problem wrong. This is also true for the upcoming EU CSAM regulation.</p><p>Once an <em>industry standard</em> is set by regulation, and which online platforms must comply with, CSAM Trust &amp; Safety managers become extremely resistant to greenlighting a different approach to tackle CSAM threats by the teams they coordinate. And with this, the antilogic rotates in circles — revealing the real origin of Trust &amp; Safety NDAs.</p><p>Since online platforms risk, with punitive CSAM regulations, having to pay highly expensive fines if new CSAM content is actually found on their sites (as ‘unknown CSAM files’ detection always happen after the CSAM files were uploaded to the platform), efficient CSAM approaches are sacrificed for the sake of compliance with the given regulation.</p><p>And this highlights how the proposed EU CSAM regulation can fail miserably, and how it merges itself with the ID1 paradox.</p><p>After all, as long as ID1 remains a CSAM Trust &amp; Safety Industry Standard, proactive CSAM detection will have no place in CSAM Trust &amp; Safety teams and CSAM numbers will keep being artificially generated (“Hey, that’s what’s in the database”).</p><p>Even though CSAM networks are highly specialized (for example, Image A may never appear together with Image B, even though this is on the surface “imaginable’’), Image A is hardly ever analyzed in its CSAM network context; moreover, the context is also not captured by ID1.</p><p>Since Image A, Image B and Image C are simply and independently added to ID1, a second state of blindness regarding CSAM databases is created — forcing CSAM Trust &amp; Safety teams to require even higher levels of training, or CSAM Threat Actors will easily bypass human eyes too.</p><p>In particular, when it comes to something as complex as CSAM &amp; Online Platforms interaction, we must be extremely careful with any ‘harmonized’ regulation. CSAM networks are not organized in the same way inside Instagram as they are in Twitter. TikTok and Snapchat present CSAM threats that are very different from each other, and so on and so forth. And this creates an issue for any regulation aimed at establishing “common Trust &amp; Safety practices” anywhere.</p><p>Additionally, CSAM networks have shown a great ability to change faster than laws do. This means that, faced with a mandatory practice, Trust &amp; Safety Teams risk being forced to comply with regulations that, soon, will not make sense anymore for detecting the latest violating content.</p><p>For me personally, I think that ID1 solutions (the very core of the new EU proposal) belong in the Trust &amp; Safety tool chest that, even though helpful, don’t deserve to be on the Unicorn’s Throne (“magical solution”) anymore.</p><p>Even though platforms such as Facebook, Twitter, TikTok and others constantly scan their platforms against ID1, the fact that CSAM networks exist within them under the radar shows us how ID1 has become outdated for its purpose as a “magical solution”.</p><p>By saying that the CSAM problem can be solved by scanning all other non-Big Tech platforms and matching them against a database of known CSAM files, the EU indirectly argues that Big Tech is not the main target of this regulation — but rather a small “Platform X” whatever that may be.</p><p>The specific CSAM threat posed by “Platform X” which MUST BE under EU jurisdiction is yet to be demonstrated while, parallel to that, a quick analysis of CSAM networks points to Asia, Oceania and Arabia as the preferred ‘jurisdictions’ for CSAM hosting — despite this, CSAM hotlines refer primarily to the EU and the United States.</p><p>This is only an example of the contradictory data we need to resolve, highlighting once more the urgent need to really research the data before proposing any solutions to solve the problem.</p><p>From a compliance perspective, the new EU proposal gives us the weird impression of a compliance program written without any risk assessment — a logical contradiction through which we derive conclusions without having ever established premises. See, for example, that the creation of the EU Center (premise generator) and the CSAM scanners are meant to be simultaneous, while the first should have been used mainly to validate the adequacy of the second.</p><p>Until now, the risk assessment documentation, the document that says ‘what is really going on’ with the targets that the EU intends to regulate is missing (e.g. what is the real problem with Instagram). The fact that we keep supporting CSAM regulations with numbers we know nothing about should cause an overall discomfort.</p><p><strong>3. If Web3 is the next big thing, what do you think of the metaverse with regard to online child safety? Is there a greater risk around child grooming?</strong></p><p>The metaverse will only surface the fact that child safety is an issue much bigger than just ‘known CSAM files’.</p><p>The hyperrealistic metaverse avenues give us the false impression that we are facing a new, or even an increased child safety threat. Instead, what we are facing is a “new wine in an old bottle.” Because the wine label is now more colorful, more visual, we tend to pay more attention to it. But that’s all it is — -a new label.</p><p>The bad news is, the prototype for the metaverse child safety threat already exists on online platforms. It’s only the lack of research and proper detection tools that keep this threat in “silent mode’’.</p><p>A quick look into online platforms’ channels will rapidly show us that CSAM Threat Actors are already explicitly organized, managing their CSAM networks together, near child profiles on some of these platforms. This is exactly what child safety advocates should be expecting in the metaverse.</p><p>What causes some degree of discomfort in the metaverse is not the graphical world it brings with it, but the fact that “CSAM networks moving in tandem” have been and are (with the ID1 as the only industry standard) well below CSAM Trust &amp; Safety teams’ radars.</p><p>The fact that metaverse worlds will allow us to actually see CSAM perpetrators walking their avatars around children’s accounts once, twice, twenty times a day doesn’t make CSAM Threat Actors more threatening — it actually makes children safer.</p><p>What was an internal log visible only to Trust &amp; Safety teams (e.g. the threat actors movement) now becomes a phenomenon easily seen by whoever is around. The question now depends on the efficiency of ‘reporting buttons’ and ‘review metrics’.</p><p>Contrary to common opinion, I personally believe that it is precisely the metaverse — because of the level of transparency afforded by its visual nature — that will take the reins of child safety for the next decade. Metaverse visuals might be the thing that instigates the “minimal duty of care” standards, and I would be very happy if that finally happened.</p><p><strong>4. Given your experience as a CSAM researcher, what advice would you give to online platforms?</strong></p><p>Child Sexual Abuse Materials (CSAM) networks must be understood as what they are: A social science phenomenon.</p><p>From a CSAM research perspective, this highlights the hidden potential of qualitative methods (“hows” instead of “how many”) such as network analysis for Trust &amp; Safety governance.</p><p>Blindly reviewing numbers hasn’t been of any help up to this point. The reason is a partnership with CSAM Researchers (external and internal ones), in order to understand what the proper research questions are, is always advisable.</p><p>If I could provide a single CSAM research insight to online platforms, I would say that the way CSAM Threat Actors interact with each other tends to provide more useful insights and a better detection system than their isolated pieces of CSAM content.</p><p>My advice would be, in this sense, to better research the birth, development, and death of CSAM networks that operate inside online platforms’ own channels — moving thus towards a proactive CSAM Trust &amp; Safety detection approach that is based on predictive metrics derived, precisely, from these social science scenarios.</p><p><strong>5. Organizations like IWF and Thorn have been created to protect children from online sexual abuse. However, for a cause that could be a global initiative, these organizations can sometimes take a proprietary posture towards their tools and data. With elimination of online child abuse being such an important problem, do you think we can get to more collaborative and distributed solutions?</strong></p><p>Proprietary postures towards CSAM tools and data can be better understood, in the case of NGOs, by a proprietary interest towards financial sponsors (hereby named “Data Sensitivity”) — rather than a wise, thoughtful decision to protect children.</p><p>Proprietary postures towards CSAM tools and data can be better understood, in the online platforms case, by the legal interest of compliance (hereby named “Data Privacy”) — rather than a wise, thoughtful decision to protect children.</p><p>Similar to the point about the proposed EU regulation, “proprietary” tools subjected to no external review also risk getting the problem wrong.</p><p>For example, even though known identifiers used by CSAM offenders and CSAM victims would have been extremely helpful for content moderators to proactively identify CSAM, Threat Actors can bypass online platform’s traditional “image-based CSAM detectors”, but “CSAM context” is hardly ever the focus of those proprietary tools.</p><p>And here we can easily see how the issue spins, once again, in circles. Because CSAM databases are such a sensitive topic, those tools become not only proprietary, but “Law Enforcement only”. But where and how does Law Enforcement work start? We absolutely need this context, and we absolutely need to integrate content moderators in this discussion. You might agree with me — as mentioned above — that with a ID1, law enforcement does not go much further — and here is how the second bottleneck is created:</p><p>Content Moderators cannot be properly trained on how to recognize a CSAM threat — as the tools are ‘highly sensitive’ — and Law Enforcement cannot proactively navigate online platforms for leads as they do not know how to find the networks. A lose-lose scenario, in which we clearly see the harm caused by three parallel worlds (NGOs, Law Enforcement Agencies and Trust &amp; Safety Groups) that should have never started working against each other.</p><p>Having Trust &amp; Safety teams interacting with CSAM NGOs only as ‘anonymous readers of public (thus filtered) research reports’ has the unfortunate effect of turning what could have been a highly impactful CSAM “Trust &amp; Safety Trends &amp; Issues” Research Report into one that hardly ever meets the actual Trust &amp; Safety Research needs.</p><p>Even though I agree that CSAM is a sensitive topic, I disagree that Trust &amp; Safety teams must be trained, for example, with the help of blind hash databases whose context they have no idea of or with the help of redacted, filtered, blurred CSAM Research Reports.</p><p>A more transparent approach to CSAM Trust &amp; Safety training — benefiting from the qualitative findings of those NGOs — is, as such, advisable.</p><p>In the era of artificial intelligence, we’ll learn how to manage and further develop synthetic datasets. A cooperative approach to training and a distributed, synthetic approach to CSAM data is, as such, my added “how” comment to a “yes” answer.</p><p><em>If you would like more information or are exploring options for AI enhanced moderation for your platform, contact us at </em><a href="/service/mailto:contact@checkstep.com"><em>contact@checkstep.com</em></a><em>. Alternatively, you can also visit our website </em><a href="http://www.checkstep.com/"><em>www.checkstep.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa633fad7f58" width="1" height="1" alt=""><hr><p><a href="https://medium.com/checkstep/experts-corner-with-carolina-christofoletti-fa633fad7f58">Expert’s Corner with Subject Matter Expert Carolina Christofoletti</a> was originally published in <a href="https://medium.com/checkstep">Checkpoint</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>