Thigh Gaps and Filtered Snaps: A Qualitative Study Exploring Opportunities to Mitigate Social Media Harm Through Content Moderation for People with Eating Disorders

Thigh Gaps and Filtered Snaps: A Qualitative Study Exploring Opportunities to Mitigate Social Media Harm Through Content Moderation for People with Eating Disorders | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Thigh Gaps and Filtered Snaps: A Qualitative Study Exploring Opportunities to Mitigate Social Media Harm Through Content Moderation for People with Eating Disorders Pranita Shrestha, Jue Xie, Pari Delir Haghighi, Michelle L. Byrne, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7136130/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Jan, 2026 Read the published version in Journal of Eating Disorders → Version 1 posted 9 You are reading this latest preprint version Abstract Background: The ubiquity of social media has increased exposure to idealised beauty standards, often unrealistic and harmful. Repeated exposure with such content has been linked to body dissatisfaction, harmful behaviours, and potentially the development of eating disorders (ED). Given the volume of content produced daily, effective harm mitigation strategies (automated or manually user-driven) are essential. Such strategies require empirically informed understanding of the underlying contexts and nuances surrounding harmful content. Objective: The study has two key aims: (1) to understand the perspectives of experts by profession and lived experience of eating disorders, on what makes social media content harmful in the context of body image and ED, including why and how this harm occurs; and (2) to explore how technology might help mitigate these effects. Methods: We engaged n=30 participants in our work. We conducted 12 interviews with experts by profession (n=2 ED support service providers and n=10 body image and ED experts), and 5 focus groups with experts by lived experience (n=18 people with lived experience of ED). Results: The thematic analysis presented six prominent themes: (1) Understanding contextual factors of social media content, (2) Contributing factors to the ED "echo chamber", (3) Challenges for content moderation in social media, (4) Needs and requirements of stakeholders for a safer social media experience, (5) Promoting diversity on social media, and (6) Perceptions regarding use of technology to mitigate the negative impact of social media. Drawing on these insights, we developed a categorisation framework consisting of eight types of harmful social media content related to body image and ED This study provides an underlying contextual understanding of harmful content related to body image and ED and highlights essential considerations for harm-reducing technologies. Conclusions: Manual content safeguards and reporting place significant responsibility on users. Through this work, we present foundations for informed rules to differentiate between harmful, ambiguous, and safe content related to body image and ED, by highlighting the underlying context. We present design insights to inform how technology might support classification systems and dynamic, adaptable automated moderation, and key considerations for reducing social media harm. social media body dissatisfaction eating disorders social media content content moderation harmful content Figures Figure 1 Plain English Summary Constant exposure to idealised beauty standards online can lead to negative body image, unhealthy behaviours and even eating disorders. To help reduce these harms, we need better ways to identify and moderate harmful content. However, first, we must understand why and how this content causes harm. We consulted n=30 participants, including interviews with n=12 professionals in ED support and research, and focus groups with n=18 individuals with lived experience of eating disorders. Six major themes emerged: the role of context in assessing harmful content, social media's contribution to an ED "echo chamber," challenges in content moderation, the need for safer user experiences, the importance of diverse representation and the potential for technology to help mitigate harm. From this analysis, we developed eight distinct categories of harmful social media content related to body image and eating disorders. These categories help clarify the various ways individuals may be negatively affected by harmful content and provide structure for future solutions. Participants highlighted the powerful influence of algorithms in promoting harmful content and called for shared responsibility among users, content creators, platforms, and policymakers. The findings offer guidance for designing technologies that mitigate social media harm. Background Each year, approximately 3.3 million individuals are affected by the physical and mental impact of eating disorders [ 1 ]. Eating disorders (ED) are severe mental health conditions characterised by unhealthy behaviours towards eating, food, and exercise, such as extreme food restriction or binge eating, relentless fixation on body shape and weight, and engagement with harmful compensatory actions like purging after eating (e.g. vomiting, overexercising, laxative use) [ 2 ]. They are closely connected with body image concerns. Body image refers to the thoughts, perceptions, feelings, and emotions about one's body regarding appearance, shape, size, and other physical attributes [ 3 ]. Individuals with negative body image can often develop disordered eating behaviours (chronic dieting, purging calories through dieting, skipping meals, or obsessive calorie counting), which in turn can lead to the development of ED [ 4 ]. Social media platforms amplify content that promotes what might be considered 'ideal' beauty standards regarding physical attractiveness and desirability, typically shaped by media, influencers and cultural norms [ 5 ]. The constant exposure of this kind of content can influence individuals' behaviour, to emulate unrealistic 'ideals.' A prime example is the rise of online 'challenges' such as the thigh gap challenge, where women strive to achieve a noticeable gap between their inner thighs when standing with their feet together [ 6 ]. Aside from this, popularisation of certain drugs for weight loss, such as Ozempic by celebrities and content creators, has also been rampant in social media [ 7 ]. In addition, broader content within the health and fitness category may promote specific ideals of muscularity or fitness to aspire to, both of which can lead to the development of unhealthy eating and exercise behaviours in an attempt to achieve this [ 7 , 8 ]. Research has additionally shown that adolescents frequently use social media as a primary source of information about diet, nutrition and fitness, often from creators who may not be credible professional sources, or who might be promoting extreme methods framed as fad diets (e.g. eating an unbalanced diet by cutting out food groups such as carbohydrates, demonisation of sugar, fasting) [ 10 ]. Numerous research studies have found a clear connection between the type of content users view on social media, body dissatisfaction and increased disordered eating behaviours [ 10 , 11 ]. With the rise of user-generated social media content, content moderation has become crucial to creating a safer online environment. Content moderation is monitoring and identifying harmful content that goes against legal, ethical and community standards and taking necessary steps towards removing such content [ 13 , 14 ]. Several popular platforms (e.g. Facebook [ 15 ], Instagram [ 16 ], TikTok [ 17 ]) additionally state that they remove pro-ED content in their guidelines. Pro-ED content is content that promotes ED as a 'lifestyle choice' rather than a severe mental condition, and reinforces and offers encouragement to partake in disordered eating behaviours [ 18 ]. The platforms heavily rely on hashtags and keywords for conducting this moderation of pro-ED content (e.g. pro-ana, ana - short form for anorexia nervosa, mia - short form for bulimia nervosa, anorexia, bulimia, thinspiration). Users who search for these terms are immediately redirected to the respective country's helpline services [ 19 ]. However, research has found that users have developed methods for bypassing hashtag moderation, by avoiding hashtags entirely or using altered versions of hashtags that can easily evade the platform's moderation systems [ 20 ]. Self-moderation is another form of content moderation where the users themselves proactively moderate and regulate the kind of content they interact with. However, to achieve this, it is important to equip the individuals with the concept of what is harmful to them and how to navigate the social media space by putting their well-being at the forefront. The transient and ever-changing nature of social media content creates further difficulty for content moderation. For example, the emerging trend of "What I eat in a day" videos, where individuals provide information about their daily meals, calorie counts or portion sizes [ 21 ]. While this type of content may seem harmless or even informative and educational, it often falls into an "ambiguous" or "grey area" category of content depending on the context in which they are presented and consumed [ 21 ]. For individuals who are at-risk of or experiencing ED, this kind of content can potentially lead to obsessive calorie tracking, food fixation and body dissatisfaction [ 22 ]. Such hyperfixation on calorie and food can be damaging for vulnerable individuals especially when social media algorithms create a loop of such content to increase engagement. Thus the benefit and risk from such ambiguous content depends on the level of self-awareness and resilience in the individual. This highlights the importance of ensuring that content moderation efforts by individuals and platforms are dynamic and adaptable so that harmful trends can be detected in the early stages. Research has highlighted that creating a safe social media environment requires a collective and shared commitment from individuals, content creators, human moderators, platforms and governments [ 23 – 26 ]. Due to the ever-growing volume of social media content, such collaborative efforts from different stakeholders can be enhanced further through automated technological systems. These can go beyond those already in place by social media platforms so as to reduce the load (on individuals, content creators and human moderators) to make content moderation choices. In the ED space, studies have explored using machine learning techniques to identify ED-related Reddit posts [ 27 , 28 ]. In addition, Abuhassan et al. [ 29 ] applied machine learning techniques to differentiate users at-risk of developing ED, from users who are merely engaging with ED-related content, based on their interactions on Twitter (now X). While much of the existing research has been concentrated on identifying and categorising content on text-based social media platforms, the growing influence and negative impact of image-centric platforms such as Instagram, TikTok and Snapchat highlight an urgent need to extend content identification and moderation to media beyond text (tweets, caption, hashtags, keywords). However, the level of subjectivity surrounding visual content (i.e. understanding what might be considered harmful and for whom) and the content's dependence on contextual nuances, creates a challenge in developing approaches to automation for detecting potentially harmful content [ 30 ]. Context can be defined as the essential background information that facilitates an understanding of the particular situation [ 31 ]. Thus, it is important to understand the context of such content to understand possible underlying subjectivities and to design and develop technology to provide better content navigation and moderation. Prior Work Current state of content moderation of social media Social media platforms each have a version of automatic content moderation [ 32 – 34 ], to remove certain content that might be against their respective community guidelines. These are usually guided by legal factors [ 35 ]. For example, Meta's community standards strictly mention the removal of content including child abuse, nudity, suicide and self-harm, explicit eating disorder (ED) content, and hate speech [ 36 ]. These platforms have AI-driven content moderation in place, which analyses the content (visual, audio) and attached textual data (keyword, hashtag, title, caption) to determine its safety [ 32 – 34 ]. If content is deemed unsafe, it is either entirely removed or its spread is limited (e.g. not shown to users under 18). Whilst this is a necessary first step in keeping social media communities safe, there is a distinct lack of transparency in the algorithms that make these decisions, and often only the most extreme content is flagged. In addition, there is a human cost associated with the development of these content moderation algorithms. Meta is currently facing a lawsuit involving human content moderators in Ghana who have reportedly suffered mental health issues after being exposed to disturbing and harmful content on social media, including extreme violence, murders, and child sexual abuse [ 37 ]. The lawsuit attributes these conditions to a lack of adequate safeguards to protect the well-being of the workers. This case highlights the urgent need for better automated content moderation work across all categories of harmful content, including eating disorders where human moderators are exposed to content such as self-starvating, disordered eating behaviours and body shaming. Specifically for self-harm and suicide prevention, when Facebook's AI algorithm flags a potential case, Facebook first provides a support option with resources. In extremely serious cases determined by Facebook's Community Operations teams, Facebook directly contacts local authorities to perform wellness checks [ 38 , 39 ]. According to Meta's official website [ 38 ], in 2018 they performed over 1000 wellness checks with the support of first responders. For ED, Facebook encourages individuals to report posts that suggest someone is experiencing an ED so that Facebook can reach out to them [ 40 ], however how this is practically implemented, and at what scale in the global reach that facebook has, is unknown. Despite these efforts from social media platforms a significant amount of ED-related content is still highly prevalent [ 41 ]. In a study by Griffiths et al. [ 42 ] the authors reported that the TikTok algorithm recommended 4343% more harmful ED content, 335% more dieting content and 142% more exercise content to people experiencing ED in comparison to people not experiencing ED. Similarly, a recent study by the Center for Countering Digital Hate "YouTube's Anorexia Algorithm" [ 43 ] reported that amongst 1000 videos recommended by the Youtube algorithm to a hypothetical 13-year old girl (a profile created by researchers), 34.4% were considered as harmful eating disorder content (breaching YouTube's safety policies) and 63.8% were videos related to weight loss or ED. As such, despite content moderation safeguards placed by certain platforms, ED-related content still reaches viewers in large amounts. Besides automated content moderation, platforms offer reporting and flagging options to the users themselves. Once content has been flagged, human moderators from the specific platform will review the content and make a final decision regarding whether or not the content breaches community guidelines [ 32 – 34 ]. Platforms also provide options to users to unfollow, mute and block certain content to enhance their ability to curate and moderate their feeds [ 44 ]. However, putting all the responsibility of moderation on the user themselves can be burdensome, as not all users have the same cognitive and critical thinking abilities to identify harmful content [ 45 ]. As such, this requires a shared responsibility for moderation between users and platforms. Technology used for mitigating harm in online spaces Numerous research studies have used Natural Language Processing (NLP) (a branch of Artificial Intelligence (AI) that understands and interprets human language) [ 63 ] to understand harmful content from textual data such as captions, hashtags and keywords to identify pro-ED content [ 29 , 63 ]. Feldman et al. [ 64 ] moved from just text-based analysis to incorporating both static image and text in social media content to classify harmful or safe content using Computer Vision (another branch of AI that understands and interprets visual information) [ 65 ]. The study developed an image classifier to distinguish between harmful (promoting ED) and safe (not promoting ED) content based on visual cues. As the study suggests, it is important to consider the visual aspect of social media due to the increase in the popularity of visual-based social media platforms. While this research is promising, Chancellor et al. [ 66 ] stressed the importance of combining automated content moderation with alternative intervention solutions, rather than simply implementing blanket bans on certain content types. The authors described that content creators find ways to circumvent content moderation strategies (e.g. by altering hashtags that might be flagged). Additionally, as social media transitions from static images to dynamic video content, understanding visual context and subtle nuances becomes increasingly important. With a growing scale of harmful content in online spaces, there has been an increase in the development of automated technologies to detect and analyse content regarding hate speech, terrorism and misinformation [ 67 ]. Google Jigsaw developed 'Perspective API' which used NLP to detect and manage toxic comments in online platforms [ 68 , 69 ]. The API categorised toxic content based on a toxicity score, which measures how offensive or disrespectful a comment is and how likely these comments could drive users away from the discussion. Aside from toxicity, there have been several studies exploring approaches towards combating fake news [ 70 – 72 ] in online spaces through automated detection and moderation. Khan et al. [ 73 ] explored the verification of news through a semi-automated machine learning tool to verify visual user-generated content by introducing multimedia forensics (technical analysis to detect any synthetic aspect or digital manipulation) [ 74 ]. Alongside the importance of detecting such harmful content, researchers [ 73 , 75 ] have stressed that it is important to understand and utilise the contextual information of such content before making any decisions. It is comparatively straightforward to identify extremely harmful content such as content that provides instruction for extreme weight loss or mocking victims of eating disorders [ 36 , 76 ]. However, for some ambiguous content, the context can differentiate whether the content is harmful or harmless [ 77 ]. This adds another layer of complexity for content related to body dissatisfaction and ED, as some of the ambiguous content might be contributing to negative behaviour formation in the early stages of ED, or perpetuating negative behaviours when an ED has already taken hold. Conversely, such content might be perfectly harmless for a non-vulnerable population. Goal of this study The goal of this study was to explore the potential role that technology can play in reducing the negative impact of social media on body image and ED. We conducted interviews with experts by profession (professionals working in a national ED service, and body image and ED experts) and focus groups with experts with lived experience of ED. Through this paper, we provide unique insights into the perspectives of diverse stakeholders within the ED space, in the context of harmful social media content for people at-risk of or with ED. These insights are intended to inform effective content categorisation and the design of future technological solutions to mitigate the adverse effects of social media. Methods We first conducted interviews with n = 12 experts by profession. Typically a minimum sample size of 12 participants is suggested for qualitative studies in order to achieve data saturation [ 78 ]. Through the interviews we narrowed down the harmful content categories and their characteristics, challenges around the identification of this content and explored potential tools to mitigate the negative impact of social media. Then, with a series of 5 focus groups with n = 18 individuals with lived experience of ED (experts by lived experience), we further investigated their views on content categories relevant to ED identified by the experts by profession, and their perceptions regarding AI technologies as a potential tool for automating harmful content characterisation. AI was specifically narrowed in on for the focus groups as this was the main category of technology that the experts by profession identified in the interviews. For the experts by lived experience, we included all individuals who expressed interest in participating in the study to capture a broad range of perspectives. Ethics We received ethical approvals from [blind for review] Human Research Ethics Committee. All participants were provided with an explanatory statement about the study and asked to sign a written consent form. Before the beginning of each session, we additionally requested that participants provide verbal consent to audio recording. To ensure the psychological safety of the lived experience participants in the focus groups, we imposed the following exclusion criteria for participation: 1) must not have an active ED, and 2) must be above 18 years old. We additionally sought in-depth feedback on all documentation and activity plans from our partner organisation (a national ED service provider in Australia), who reviewed the terminology and tone of language that we used and advised on the appropriateness of activities. They also provided us with necessary helpline services and instructions that we could provide the participants at the beginning of the focus group, to ensure adequate support provision should a participant become uncomfortable during the session. Finally, a [partner organisation] member attended the first focus group, as an observer, to ensure that our activities and the research team's facilitation were adequate. Two research team members facilitated each focus group session. Facilitators had extensive experience conducting qualitative research within the themes of mental health, ED, and digital innovation. Recruitment We recruited n = 12 leading body image and ED experts across Australia, by leveraging professional networks within the research team and a snowball sampling approach (i.e. the professional networks of existing participants). We emailed advertisements to gauge interest and interested parties were provided with the explanatory statement and an opportunity to ask questions to the research team. Interviews were scheduled at a convenient time for participants, with written consent obtained prior to each session. Participants received an AUD 20 shopping voucher as thanks. For the focus groups, we recruited n = 18 individuals with lived experience of ED through [partner organisation’s] lived experience network, which encompasses people from Australia with a history of ED. Recruitment materials highlighted particular interest in speaking to traditionally underrepresented people in ED research (i.e. males, people from culturally and gender-diverse backgrounds). Interested participants filled out a Google form and were provided with the explanatory statement and an opportunity to ask questions to the research team. Focus groups were arranged at a convenient time for participants, with written consent obtained prior to each session. Participants received an AUD 60 reimbursement as thanks. Interview with experts by profession We conducted the interviews via Zoom which lasted approximately 1 hour. It is to be noted that when we conducted the interviews, the Australian government had not yet passed the bill for banning social media for users under 16 years old [ 79 ]. The participants' demographic details are provided in Table 1 . The interviews focused on exploring four key areas: 1) Participants' experience and the specific populations they work with; 2) Influence of social media on body image and ED; 3) Characteristics that might be useful for identifying social media content as harmful, ambiguous and safe; and 4) Potential solutions (digital or non-digital) to mitigate the negative effects of social media on body image and ED. The interview guide can be viewed in Appendix A. Through a preliminary analysis of the interview data, supported by a review of existing literature, we generated eight content categories relevant to ED: weight loss (including content influencing changes to body shape and size), weight loss or performance-enhancing drugs, ED recovery, cosmetic surgery and other minimally invasive cosmetic procedures, beauty tutorials, food and nutrition, exercise, and body checking (i.e. where the creator is checking their body either in the camera, on weighing scales, or by measuring parts of their body). Table 1 Demographics of experts by profession (interviews) [Note: Participants self-identified their ethnicity or cultural background and this was not standardised by the research team] Code Gender Ethnicity or Cultural background Age Lived experience of ED Lived experience of body dissatisfaction Int1 Female White 25–34 No No Int2 Female White 25–34 No No Int3 Female White 25–34 No No Int4 Female White 35–59 No No Int5 Female Anglo/White 35–59 No Yes Int6 Female Lebanese 25–34 No No Int7 Female Bengali 25–34 No Yes Int8 Male White Australian 25–34 Yes Yes Int9 Male White 25–34 No No Int10 Gender-diverse (Non-binary) Australian 25–34 No No Int11 Female Australian 25–34 Yes Yes Int12 Female Australian 35–59 No No Focus groups with experts by lived experience of ED We conducted a series of 5 focus groups each lasting 1.5 hours via Zoom, with n = 18 participants. We conducted focus groups 1, 2 and 3 with females (n = 3, n = 5 and n = 6, respectively), focus group 4 with gender-diverse participants (n = 2), and finally, focus group 5 with males (n = 2). We formed these groupings to foster a comfortable discussion environment. We explicitly asked participants from focus groups 4 and 5 if they would prefer to be placed in a mixed or gender-specific grouping and they chose gender-specific. We have provided the participants' demographic details in Table 2 . The focus groups aimed to examine the role of social media in developing ED and explore potential solutions to make social media safer. We used Miro boards (an online whiteboards that allows people to collaborate in real-time using diagrams, sticky notes and other tools to write down and organise ideas) to engage with the participants. The interviews with experts by profession fed into the planning of activities for the focus group. For our first activity, we asked the participants to discuss the role that social media played at different stages of ED: early stage (i.e. where disordered eating patterns and challenging behaviours might be developing), late stage (i.e. where the ED is fully embedded and might require formal intervention), and during recovery. We gained a crucial and nuanced understanding of how certain content impacts users differently on where they are in their eating disorder journey depending on their vulnerability and context. The participants discussed both positive and negative aspects of social media for each of these stages as shown in Fig. 1 . For our second activity, we asked participants to discuss as a group and create 'We need' statements for a hypothetical manifesto positioned to the government to advocate for safer social media platforms for those at-risk of or experiencing ED (inspired by methods in [ 80 ]). The participants then prioritised and selected their top three statements as a group. Through this activity, we aimed to understand what kind of harm was considered as a priority by experts by lived experience, and what they considered as safety and support. By taking into account what participants say they need to feel safe and supported, we can design moderation tools that are aligned with real-world experiences. During the interviews with experts by profession, over half of participants provided ideas for the potential solutions to mitigate the negative effects of social media on body image and ED which revolved around the use of Artificial Intelligence (AI) (e.g. use of chatbots to promote body positivity and provide automatic critical thinking tasks when certain images are detected in the social media feed). Thus, during the focus groups, we decided to explore the participants' views on AI and the potential use of AI to create safer social media. Building on this, for our third activity, we asked participants to imagine that their manifesto (created in the second activity) was accepted by the government, who had then created an AI tool embedded in social media. This activity was intended to be a provocation to yield open discussion and prompt design insights. As such, there was no description provided about how the tool functioned. This loosely followed the values of invisible design approaches, an approach which is used to generate ideas and explore insights as a form of concept development [ 81 ]. We asked participants to consider AI use cases, how it made them feel, how it made their life easier, potential challenges, and prominent supporters and critics. Finally, participants created rules for the AI to guide content moderation by identifying and categorising harmful social media content for the eight content categories, which we derived from the interviews with the experts by profession. They were asked to list 'Must not show' (prohibited content for social media) and 'Can show' (acceptable for social media) content for each category, helping us understand their views on what content is harmful or safe for social media. The focus group guide can be viewed in Appendix B. Table 2 Demographics of the experts by lived experience of ED (focus group) [Note: The participants self-identified their ethnicity or cultural backgrounds and this was not standardised by the research team] Code Gender Ethnicity or Cultural background Age Religion Type of Eating Disorder (ED) FG1P1 Female Australian 25–34 NA Anorexia nervosa FG1P2 Female Australian/ New Zealand nationality of Scottish descent 25–34 Anglican Bulimia nervosa FG1P3 Female European 25–34 NA Anorexia nervosa FG2P1 Female Malaysian Chinese 25–34 Christian Anorexia nervosa FG2P2 Female European Australian 10–24 Agnostic Anorexia nervosa FG2P3 Female Anglo-Australian 10–24 Christian Anorexia nervosa FG2P4 Female European 35–59 Catholic Anorexia nervosa FG2P5 Female Indian 10–24 Atheist Anorexia nervosa FG3P1 Female Caucasian 25–34 Atheist Anorexia nervosa FG3P2 Female Australian 25–34 None Anorexia nervosa and Binge eating disorder FG3P3 Female Australian 25–34 NA Multiple types FG3P4 Female White British/ Australian 25–34 None Other specified feeding or eating disorders (OFSED) and Orthorexia FG3P5 Female Caucasian Australian 25–34 NA Anorexia nervosa FG3P6 Female Iraqi Muslim 25–34 Islam - Shia sect Anorexia nervosa + Avoidant/restrictive food intake disorder (AFRID) FG4P1 Gender-diverse (Transgender man) Australian with a Greek cultural background 25–34 None Anorexia nervosa FG4P2 Gender-diverse (Non-binary) Maori, European 25–34 None Bulimia nervosa FG5P1 Male Sri Lankan, Australian 25–34 Buddhist Anorexia nervosa FG5P2 Male Caucasian 35–59 None Anorexia nervosa Data Analysis Across the study, we had 19.5 hours of audio-recorded data. The first author transcribed all the audio recordings and de-identified the transcripts. After the completion of the interviews with experts by profession, the first author used an inductive thematic analysis method [ 82 ] to generate themes and codes at the paragraph-to-sentence level without any pre-existing theoretical framework. NVIVO was used to code the transcripts. The first author generated the initial codes and the remaining research team familiarised themselves with the data from the experts by profession and reviewed the initial codes. After iterative coding, scoping of the existing literature and numerous discussions a set of content categories was generated. These categories were further explored in the focus groups with experts by lived experience. After the completion of the focus groups, the first author again used inductive thematic analysis [ 82 ] to generate initial codes and themes which were reviewed and discussed in an iterative process by the research team. Any discrepancies were resolved through discussion until a consensus was reached on the final significant themes. Data from the existing literature, interviews and focus groups were triangulated and are presented as such in the findings [ 83 ]. The interviews provided in-depth academic and professional knowledge and reasoning behind harmful content categorisation, as well as insights around opportunities for technology to mitigate the negative effects of social media. The focus groups then added a crucial layer of nuance by providing personal perspectives from those with lived experience of ED. The triangulation confirmed emerging themes and increased the credibility of the findings [ 84 ]. Results In this section, we will consolidate the findings from interviews and focus groups. For easy identification, we have added prefixes to the participants of interviews and focus groups: interviews (Int) and focus group (FG). Throughout our findings, we refer to participants from interviews as experts by profession and focus groups as experts by lived experience, to further add clarity regarding where data was derived. Six core themes from across the data are reported below: 1) Understanding contextual factors of social media content, 2) Contributing factors to the ED echo chamber, 3) Challenges for content moderation in social media, 4) Needs and requirements of stakeholders for a safer social media experience, 5) Promoting diversity in social media, and 6) Perceptions regarding use of technology to mitigate the negative impact of social media. Understanding contextual factors of social media content The experts by profession stressed that understanding social media content can be nuanced and subjective, which becomes a challenge in differentiating between harmful and safe content; "the fact that it's not black and white. [...] People might have different perspectives on whether something says red [harmful] or amber [ambiguous] or even green [safe]" (Int12) and "I guess this is the problem because it wouldn't be harmful for everybody [...] So, there would be a lot of people that look at that content and would be absolutely fine and not harmed at all." (Int2). Int2 suggested that to overcome this challenge, the potential solution could be to separate harmful content categorisation for different population groups: "whether we categorise as harmful for everyone or harmful for people 'at-risk'. So, for example, people with ED, people at-risk of ED, or people in recovery could be part of that kind of 'at-risk' group, and then you have kind of a general one" (Int2). Despite nuances and subjectivity, there was clear value in consolidating the dispersed ideas of what constitutes 'harmful' content and developing a standardised, consensus-based list "Thank you for doing this work. I do think it's important to pull it all together [..] if we can have a consensus on this list that's going to be really powerful" (Int5). Participants broadly discussed a range of different social media content types, from generalised content that might be considered normal viewing on social media platforms (e.g. healthy eating recipes that provide nutritional guidance, exercise videos, and beauty tutorials) to more obviously challenging content (e.g. the promotion of weight loss drugs and cosmetic procedures, very underweight creators performing body checking behaviours - weighing, measuring, pinching parts of their body, or generally looking for signs of visible fat). Concerning more generalised content, which could be considered ambiguous in its impact, the experts by profession agreed that the intention of the video was vital: "They [fitness influencers] are more ambiguous because while people may take that as inspiration and use that for a healthier lifestyle to get more in shape and to have just a better physical health, it can be negative for those who are obsessed with exercise and engage in excessive exercise that is not necessary for them. And a lot of that exercise is driven by appearance more so than driven by health" (Int7). The experts by profession suggested that ambiguous content starts moving towards harmful when it starts focusing towards comparisons like 'before and after': "we find that [before and after] really toxic obviously because they're saying that in the first image they're not worthy. They're not worthy unless they lose weight and head towards that after image" (Int11) and content promoting very low-calorie diets: "Some content might have meal plans that follow really low-calorie meal plan [...] if people are encouraged to follow that plan to lose weight or because they're not worthy of their current appearance, that can be really damaging" (Int11). In addition, accompanying textual representations of elements such as the creator's weight or body measurements, the calorie content of foods, and calories burned, were often seen to be the potentially harmful element, even if the overall tone of the video was more positive: "Anything that shows numbers like if someone talk about how many kilograms they weigh and what their goal weights are, or showing how many calories they might be eating, or how many grams of sugar" (Int5) and "I think stuff using quantitative numbers, things like that should probably be banned. Anything where someone says their weight or calorie limits should be banned" (FG2P3). The experts by lived experience further clarified why showing numbers in such content can be so damaging: "As somebody who weighed their food and counting calories for years, it took forever to get out of that[...] a lot of the times these people are guesstimating calories. It just doesn't need to be there" (FG2P5). Perhaps the most extreme form of content was that depicting body-checking behaviours from creators who would be considered underweight: "encouraging people to take their headphone cord and see if they could wrap it around to their stomach and if they were able to reach the full head headphone cord that was good, they were skinny" (Int11). 'What I eat in a day' videos (a common trend on social media depicting videos around what a person eats throughout the day) were seen as particularly challenging across the entire cohort of participants: "because everyone's different. What might be sufficient for me to eat in one day may not be sufficient for someone else to eat in one day. It doesn't take anyone's history or medical conditions or cultural aspects in consideration" (FG3P6). Participants stressed that any type of diet plan or health advice should come from experts with proper credentials, or have a disclaimer that it is not applicable for all: "even if someone is qualified to be giving that advice, they don't know who's going to be taking it in" (Int11); and "a lot of these health gurus that might not even be trained in certain areas about food or nutrition and like targeting a lot of vulnerable people to buy certain products which can also lead to body image concerns" (FG4P1). The experts by profession additionally highlighted several other significant body-related trends, such as the 'A4 Waist Challenge': "A trend that involved getting an A4 piece of paper and holding that up in front of your body as well. And if your body was larger than that vertical, a four piece of paper, you would be shamed. If it was smaller, you would obviously be praised and considered that's an ideal" (Int11) and the promotion of thigh gaps as a body ideal: "Thigh gaps, which I think now has kind of transitioned into legging legs. [...] do you have like a big thigh gap, and they're just like straight twigs?" (Int1). Content promoting weight loss drugs (like Ozempic and Mounjaro), or performance-enhancing steroids, to aid in increasing muscle mass, were seen to be extremely harmful and often facilitated the spread of misleading information, that could put one's health at risk: "If someone, for example, is saying "Oh!! take Turkesterone [help increase muscle mass and strength], it'll boost your testosterone, you'll get mad gains", there's zero evidence for the safety and efficacy of that. Therefore, that is misinformation, and that kind of content is harmful either on misleading ground or because it's intensely negative" (Int9). The experts by lived experience were vehemently against showcasing any advertisements related to weight loss drugs: "It will put them [any viewers] at risk [...] I don't know why an advertisement would be needed" (FG3P2) and "I would love to see that just not exist on social media personally" (FG4P2). When it came to content discussing ED recovery, there were mixed views. Participants made it clear that ED recovery content showcasing medical components or hospital settings should be considered harmful, as it provides a source for comparison of symptoms. If not presented carefully, this could be considered triggering, or indeed a motivator for people at the earlier stages of their ED: "Any content that is based in a hospital setting or implies that someone has to go to a hospital for their ED to be taken seriously, that would obviously be toxic" (Int11) and "They're doing XYZ, so this means that they have an ED, whereas I don't have XYZ symptoms or I don't have those food rules. So, therefore maybe I don't have an ED. I'm just spiralling for no reason and I don't actually have to get help " (FG2P5). Finally, a surprising source of ambiguous content that was discussed was beauty tutorials. While beauty tutorials don't directly impact ED, they can cause body dissatisfaction and strengthen the 'idealised' concept that can cause serious harm to society: "We've seen an influx of 12-years-old going into Mecca and Sephora and wanting to buy retinol [anti-ageing product] and ingredients that are just way beyond their age. [...] We often think about body image as being something from the neck down. But, I think social media has tried to include everything" (Int4) and "Definitely harmful, obviously for emphasising that appearance is super important " (FG3P3). The drive to change the way one looks, to attain an 'ideal', was also seen to potentially lead to more drastic steps, such as unnecessary cosmetic procedures: "content that encourages dieting or, things like unnecessary cosmetic or medical procedures to appear a particular way" (Int4). The experts by lived experience stressed the exploitative nature of this type of content: "it's driven by exploiting someone's emotional state. It's all hype and bubble [...] it shouldn't be in the hands of someone who's making profit" (FG5P2). Contributing factors to the ED "echo chamber" An ED "echo chamber" refers to a digital space where content normalising and promoting disordered eating behaviours are continuously shared, such that the individuals in the space are repeatedly exposed to content which further reinforces their disordered eating beliefs and behaviours. Participants in all phases discussed the development of an echo chamber of sorts, created through continuous viewing of certain content types, which served to normalise unrealistic body standards and feed into existing belief systems: "it's going to create these appearance norms where people think 'Ohh!! Everyone looks like this and I should look like this'. So it's adding to these pressures, both in terms of the body size and facial appearance" (Int1) and "The more we're exposed to them [unrealistic ideals], I think the more they're perceived as normal" (FG1P1). The experts by lived experience brought a new perspective around how certain types of content could create a sense of competition in individuals at-risk of or with ED: "There is that culture on social media of competition to see who can be the skinniest, who can recover the fastest, or who can do whatever. It's a real competition to keep up with those things" (FG2P4). The echo chamber of ED content starts to blur the boundary between reality and harmful ideals, with filters and photo-shopped content playing a leading role in perpetuating unrealistic standards: "As a society as a whole, the less realistic, the less attainable, the more photo-shopped, the more filtered tends to be unhelpful because it's portraying something that's not actually real or achievable for the vast majority of people" (Int12). It distorts reality to the point that, when individuals come face-to-face with reality, they cannot accept it: "I got at a certain point, I kind of got used to the filtered version of me. So when I looked in the mirror and saw the real version, it was quite upsetting for me and also that sense of discomfort of not being able to post an unfiltered picture when you get used to the filtered version of yourself" (FG4P2). The disconnect between filtered social media content and how individuals appear can lead to people feeling as though they don't measure up: "with filters to hide cellulite or wrinkles or whatever it might be. I think for those vulnerable people continually seeing these types of images makes them feel as though they're sort of outside of the box if they don't fit that mould" (FG1P3). The inherent harm of social media algorithms within this context further highlighted this point. If viewers are watching certain types of harmful content, algorithms are learning this pattern and consequently exposing them to similar content, creating a feedback loop that intensifies harmful narratives: "It's not just the content, it's the algorithms that recommend and deliver content and the ways that people engage with that content, whether consciously or unconsciously, that makes it more likely that the content will be delivered to them" (Int9). Existing biases in the algorithm itself can also serve to further strengthen the likelihood of harmful content reaching certain demographics: "the algorithm is gender biased, so it will show young girls who have not searched for anything or followed anything in relation to diet and nutrition, but it will show girls diet content very early on in their journey on the platform " (Int5) and "there have been some suggestions that there is some dark stuff happening around censoring non-white creatives" (Int1). Trying to combat this echo chamber, created by the algorithm, was something the experts by lived experience found significantly challenging, particularly in the recovery process. The continuous bombardment of such content threatened, and potentially penetrated, the shield they had formed against these concepts: "There is this whole thing of constant everyday battle with your algorithm. As soon as you open anything [...] and no matter what you're doing, you're still opening up feeds that are going to have things in it [harmful], so you're still put in that space where you're having to combat and filter and challenge all of these messages. So, there's still a big drain that comes from it" (FG1P1). For some, this led to a withdrawal from social media altogether in an attempt to protect their recovery. As discussed by the experts by profession, comments left by other social media viewers on creators' content were often seen to be as harmful as the content itself: "I find with a lot of toxic social media content, especially on TikTok, you'll often get comments as well that are just as toxic, if not more toxic than the actual video." (Int11). These comments can amplify negative views surrounding body image, acting somewhat as evidence that a person is not good enough in the eyes of society: "the comments that come through is a very clear indicator of the public perception or opinion on whatever this main topic is" (Int6) and "you're seeing someone else getting negative comments who doesn't make those ideals and that can make you feel negative, even if those comments are not directed at you specifically" (Int10). Even when the content itself was seen to have positive components, such as promoting encouragement to live a healthier lifestyle, the negative comments were seen to overpower the positivity of the content itself. This phenomenon was particularly notable in content made by creators living in larger bodies: "The post that generates those kinds of conversation can be a positive post like it could be a fat person enjoying life or a fat person getting married or succeeding, but there will still be comments underneath that are fat phobic, negative and abusive towards that person" (FG4P2). Challenges for content moderation in social media The experts by profession expressed frustration over their experiences with content reporting, with the manual scouring and flagging of content a resource-intensive process: "And so they [social media platforms] come and say we need more information, that is really time and resource heavy. It takes a long time to then go back through especially if you need to go back through lots of different videos and comments. [...] Certainly, the time and manual aspect of it would be a challenge" (Int11). It was often the case that social media platforms refused to remove harmful content, citing their platform guidelines as a justification rather than trusting the expert opinions of ED specialists: "Someone in our community found this person on TikTok who was doing these years of body checking, and this person was really, really underweight [...] quite a lot of the comments were full with people saying 'give me your diet tips' - that sort of thing. And her account was just dedicated towards these body checks. However, she didn't mention eating disorders in the caption and she didn't talk about eating disorders at all. It was only the people commenting that mentioned eating disorders. Since she didn't mention eating disorders, TikTok said that it did not go against their guidelines" (Int11). Of course, in many cases, particularly with more ambiguous content, creators are not aware that their content could be harmful to people experiencing or at-risk of ED. However, in others, creators deliberately try to evade social media safeguards to keep their harmful content online, through methods such as mutation or avoidance of hashtags: "People mutate those hashtags. So '#anorexia' might now be displayed as [an0r3xia], but the 'O' is a zero or the the 'E' is three" (Int11) and "A lot of people on Twitter will post what is ostensible 'thinspiration' and not hashtag it with particular things because they know that if people see it, who might not have an active eating disorder or be actively kind of restricting, they'll report it and they'll get removed." (Int8). Even banning certain harmful hashtags does not guarantee proper blockage of harmful content. The experts by lived experience drew this concept out further, sharing incidents of 'hashtag hijacking': "I think it can be difficult when hashtags that are meant to be helpful, especially for hashtags saying recovery for eating disorders are hijacked by influencers or people promoting things like weight loss. [...] Concepts of 'intuitive eating' is like 'intuitive fasting' now and that's really unhelpful" (FG3P4). There is a distinct tension surrounding social media platforms in regards to maintaining a safe online environment for viewers and not implicating freedom of speech: "I think the freedom of speech is definitely a big one and definitely something that I've felt from the respective apps [social media platforms]. That is a very big concern of theirs and they want to keep that in check so they're not censoring their users to such an extent where they will go to a different app to share their story and talk about the same things" (Int11). This freedom of speech differs from country to country, which might pose a challenge for content moderation more broadly: "There's a cultural element to that as well, and that a lot of these companies are American-based where the cultural component of freedom of speech is so strong and powerful. I think in Australia, we're more used to being censored, and we're more comfortable with the benefit that can have for protecting vulnerable people. But I think in America the swing is towards that freedom of speech element" (Int12). However, the experts by lived experience were adamant that 'freedom of speech' should not be an excuse provided by content creators and platforms to push harmful content, that could ultimately affect someone's life: "All the pushbacks towards governments trying to introduce greater punishments and regulation for these platforms in the name of freedom of speech, but the big argument is if it impacts people's mental health badly then you can't cause millions of people [harm] by affecting their mental health and get away with it completely. There needs to be some sort of accountability for both the creators and platforms" (FG2P3). Participants felt that the corporations and content creators who are beneficiaries of diet culture and societal beauty standards, and who rely on advertisements and content reach to promote products could potentially be the barrier for resistance to content moderation: "I think a lot of people would have difficulty with being told that they can't post certain things or seek out certain information. [..] I think the diet culture is so prevalent that fills the pockets of industries. Individuals will struggle to adhere to the rules" (FG3P1) and "People, organisations and companies whose income and profit is based on promoting harmful content on social media [...] I think would be quite a critic here" (FG5P1). Needs and requirements of stakeholders for a safer social media experience Participants in all phases mentioned the need to bring government and social media platforms into the agenda to make social media space safer. The needs ranged from making the government aware of the situation "can we go to policymakers and say that this type of content is either safe or ambiguous or might help to mitigate some of the onslaught of idealised content that's pushed to these demographics" (Int4) and making it compulsory for platforms to open their API (Application programming interface) for research "By giving us access to things like the API and, within reason, how the algorithms work, research can be more specific... higher quality; and policies that are implementable and useful and that don't put undue burden on platforms can be brought into being" (Int9). Additionally, Int9 also discussed how researchers could aid the government in bringing platforms into the loop, by providing them with solid evidence: "The more specific we make our research and the better we can articulate mechanisms, the easier it is for policymakers to move forward, because if we present them with findings that are tantamount to, 'social media, on average is weakly positively correlated with body dissatisfaction' - What can they do with it? They can't ban social media. They don't know what to even tell the social media companies. So that's the best thing we can do is to push ourselves for methodological complexity and specificity in the research" (Int9). All participants agreed that there was value to social media, and future interventions needed to work with, rather than against, the benefits it could provide: "They're [young people] born in the age of social media. We just need to work out" (FG2P4) and "because then you miss out on the positives that social media provides as well. [...] you lack that connection with certain people and stuff like that" (FG3P1). The experts by profession, in particular, highlighted how responsible content creators were useful vehicles for promoting positive behaviours: "They [responsible fitness content creators] do have ideal bodies themselves [...] but they are doing it in the most healthful way that can be done. And what I like about that content is that it's practical and has a really significant reach" (Int9). The experts by profession also emphasised the necessity of collaborating with content creators to promote responsible content generation: "I think that would be really great if we could see a culture shift amongst influencers and content creators as to what they're posting, how they're posting it and if it's done in a safe way" (Int11). However, the experts by lived experience described that current safety measures that content creators often provide were not enough: "I agree with filtering out things before they get to being in front of the person cause things like trigger warnings and blurred posts don't do anything" (FG3P1). Promoting diversity on social media Participants in all the phases stressed that it was necessary to see more diversity in social media content, in terms of body shape, size, colour and gender: "I think it's healthy to see diversity of body shape and size" (Int12). Diversity is essential to normalise realistic body standards and create a sense of belonging and connection for the viewers. This sense of belonging is particularly significant for ED recovery: "I didn't see a single person of colour talking about their ED. And I think that just definitely felt like I was alienated even more" (FG2P5) and "I wasn't able to see myself represented anywhere in stories of hope. So, then I didn't think there was hope when I was in recovery" (FG4P1). This concept of diversity needs to be more broadly expanded towards underrepresented populations in the ED space (culturally diverse, males, gender-diverse and differently-abled people): "it was females who were like White and upper class and that they were the people that I'd see share their stories, but I never really saw any diversity of people with disabilities or men or people with different gender and sexuality or different race or homeless or anything like that" (FG4P1). The experts by lived experience also shared how not being represented in promotional materials for support services on social media could create a barrier for seeking help: "A lot of body neutrality posts are geared towards young women. I think a lot of support services are geared towards young women as well [...] I held a lot of shame around accessing services because I was a guy who was experiencing something like that" (FG5P1). The lack of representation becomes further complicated when intersectionality comes into play: "When you're a person who experiences multiple types of disadvantages. So, for me, it was my body size, you know, being a fat person, but other things like my ethnicity, my facial appearance my body shape, socioeconomic and all other things, I think it almost placed a greater pressure on me to try and be beautiful because I can't change these others things about myself [...] it felt like the only the only way I would ever be able to join the realm of the acceptableness" (FG4P2). Perceptions regarding use of technology to mitigate the negative impact of social media When asked about the technologies that might aid in mitigating negative impacts of social media, participants discussed the value of automation to support the time-intensive process of moderation: "AI could go in and kind of just flag anything that could potentially harmful, so that as a moderator, you're not having to constantly go through because it's time consuming" (Int4) and "If it [AI] could identify that kind of content [harmful] and remove it completely" (FG2P1). The experts by lived experience additionally suggested an option for filter removal: "remove like filters that alter your body and appearance so then everyone comes through as who they actually are" (FG4P1). The experts by profession also valued the potential screening and tracking capabilities of technology, for automating the detection of viewers' behaviours: "detecting that there's an issue for someone by looking at their patterns of usage of online social media" (Int6) and identifying patterns of viewing harmful content: "If you can see this, you can present this to the user and you can try to deliver notifications that move people somewhere else" (Int9). The experts by lived experience had similar thoughts: "screen if someone is searching for particular content [harmful] and maybe alert a psychologist that they might need support or follow-up" (FG2P2). For the experts by lived experience, having a level of personalisation was vital to support individual needs: "like having some personalisation so that it can be customisable because it's very different for everyone" (FG4P2), suggesting a form of of feedback system to support better recommender systems: "Maybe if afterwards or during [watching content], you've got the option of AI asking, 'Is there something you didn't like in this video? That you want us to hide in the future'." (FG1P3). In their view, using technology with added safeguards would give them back control and make them feel safer: "I think it would make me feel more willing to spend time online and engaging in online spaces. There's so much stuff that I don't have control over, and it's not like I don't know." (FG1P2); "Probably feel more at peace, like less consumed by like all this misinformation and be able to trust myself more" (FG4P1); and "Well, mine would probably be AI that alters the algorithm to show people content that uplifts them" (FG4P2). Participants also saw this as a chance to use algorithmic manipulation to populate social media feeds with positive and happy content: "you limit to only the stuff that brings you joy, the stuff that captures your attention or keeps you watching that brings you joy and we're just going to show you that" (Int1); and "Anyone who’s coming up with this thing [AI solution], would have spent enough time to work out the plotted chart of where it leads to. So, if that information is available to people to know where it’s going and what is the intention behind it" (FG5P2). However, the experts by profession had some concerns regarding the technology itself in terms of its performance, due to the subjectivity of what might be considered harmful: "I worry that the machine learning wouldn't be able to distinguish between stuff that was harmful and not harmful because there's such a nuance to that. I think probably as I've been talking and it's very hard to quantify what is actually harmful or not" (Int10). They emphasised the need for human intervention in specific scenarios, rather than relying solely on the technology to take specific actions: "I guess ideally if someone is expressing very troubling things to a chatbot [technology], it should sort of connect them with someone that they can speak to, like a real person" (Int3). Int3 further added that technology has to be appropriately evaluated and validated before use: "As long as there was lots of piloting of it, then I think I would feel OK" (Int10). There were also some concerns from the experts by lived experience regarding using AI in clinical settings: "I have concerns around how AI is used in diagnosis and treatment. But, I think I would perhaps look at AI being like the prevention side of things and identify the systems that currently exist that are creating harmful environments for people around their body image and ED, or maybe at the risk of ED" (FG5P1). Discussion Our study aimed to explore the different contextual factors within harmful social media content for people at-risk of or experiencing ED, inform content moderation strategies and assess the potential role of technology in reducing the negative impact of social media. Our discussion is structured as follows. First, we discuss how there is a need for shared responsibility between different key stakeholders and how technology can support that shared responsibility in content moderation. We then describe opportunities for integrating diversity and culture into social media, to create a more inclusive and diverse environment. Finally, we explore how the use of understanding these nuanced contexts could support novel approaches towards the design of future interventions to protect social media users experiencing, or at-risk, of eating disorders from harm. Principal Findings Need for shared responsibility in content moderation Our findings indicate that filtering harmful content should not fall solely on individuals with ED or their supporters. Shared responsibility from users, creators, platforms, and government is essential. There is potential in leveraging the inherent design of social media algorithms, which optimises user engagement by curating tailored content. Algorithmic manipulation might involve viewers proactively taking steps to curate their social media feeds through 'digital pruning' [ 85 ]. Unfollowing, muting or blocking harmful accounts and content, and critically thinking about harmful content and its potential impact is an essential aspect of media literacy [ 86 ]. By deliberately following content or creators that don't focus on appearance (e.g. instead focusing on content that aligns with one's hobbies and interests) [ 87 ], users can reduce the recommendation of 'idealised' content and provide a greater sense of control and agency. There is a significant body of work which has developed media literacy programs [ 50 , 88 ], however as our participants stressed, focusing on only the users to curate their content and think about harmful content in a critical way is not enough, as this runs the risk of exposing potentially vulnerable users to harmful content in the first place, which is a known risk factor for ED formation and exacerbation [ 89 , 90 ]. To ease the burden on the individual, we could potentially use automated technology such as mobile sensing approaches and automated monitoring of social media behaviours (e.g.Facebook's suicide prevention algorithm which browses through user posts to detect potential suicide risks [ 38 , 39 ]). The experts by lived experience in our study mentioned that they were not apprehensive to use automated social media monitoring if it meant that they would be safe from harmful content. Similarly, Vega et al. [ 91 ] reported that their participants with binge eating and bulimia showed high levels of acceptance for automated sensing approaches and engagement with reflective activities and contextual logging. The findings of this study align with our results, highlighting the potential of context-aware systems in monitoring and reflecting on social media behaviours which could be a promising area for further research. The Australian government is acutely aware of the dangers of social media for children and young people, which can be seen through the ban of social media for children below 16 years [ 79 ]. While this ban emphasises a responsibility for social media platforms to actively engage in safeguarding vulnerable populations from online harms, it also underscored the broader need for shared responsibility. Aside from age-based protections, it is crucial to address the needs of other vulnerable groups such as individuals who are at-risk of or experiencing ED and who may fall outside the ‘under 16’ age bracket. Ensuring online safety for all users requires a collaborative approach, where both government policy and platform accountability come together to address the online harm. As our findings showed, the reality is that despite features like unfollowing, muting and blocking in platforms, loopholes in platform guidelines still allow some creators to bypass moderation systems. These bypasses can lead to viewers being recommended similar content again [ 20 ]. As a solution, platforms could provide a 'reset' algorithm option, which would help the algorithm unlearn patterns and behaviours of the viewers and start afresh. For example, if an individual is diagnosed with ED or finds themselves in a period of ED relapse, then they can reset their social media algorithm rather than cutting off social media altogether. As expressed by our lived experience participants, this could be especially valuable during the recovery stage of ED, when users are particularly vulnerable to the impacts of harmful content. One factor that future researchers conducting content moderation work need to be mindful of is that moderation efforts often face criticism under the banner of 'freedom of speech'. There is an ongoing battle between 'freedom of speech' and public safety and well-being. Kozyreva et al. [ 92 ] explored the critical factors that can tip the scales between these conflicting interests: the extent of harm, frequency and repetition of conducting harm, and the content category. Creating blanket bans on content categories is highly infeasible as without proper critical thinking skills regarding its harm, users will figure out ways to circumvent blanket bans [ 66 ]. Such blanket bans could potentially censor recovery content that could be highly motivating and useful for individuals at-risk of or experiencing ED [ 93 ]. Instead, future research could focus on supporting users and moderators (like our experts by profession) to capture relevant data and automate some of the time-consuming processes such as personalised report generation for findings, focus on automated data collection rather than self-reporting, so that they can present solid cases to social media platforms for required content bans. This type of supported data collection may also prove useful for generating reports that drive government support for restrictions on certain social media content. For example, in a report from 'Reset Australia' [ 94 ], the authors presented an experimental study where paid-for-advertisement approval systems from various platforms approved their pro-ED advertisement. Such easy bypasses expose vulnerable viewers to harmful content, and studies like this show how easy it is to slip through existing content moderation processes. Building technologies that support evidence collection (e.g. tracking and analysing approval processes for ads, interrogating and analysing platform guidelines and extracting comments from social media posts) could be a significantly valuable contribution, providing evidence that would support governments to take action against social media platforms that allow such content. Enhancing diversity to break out of the echo chamber Our findings showcased the importance of bringing diversity (in terms of body shape and size, culture, gender, and ability status) into social media feeds to help support an escape from the echo chambers that social media algorithms create. Enhancing diversity not only serves to blur the lines between what is considered 'ideal' and the reality of how society is made up, but also helps people who might be outside of the stereotypical white, female and anorexic vision of an ED [ 95 ] feel connected during ED recovery. However, like many AI-driven algorithms, social media algorithms are inherently biased. The findings from our research resonate with multiple research studies, which have found social media algorithms to be biased towards creators with disabilities [ 96 ], larger bodies [ 97 ], from different racial backgrounds [ 98 ] and from LGBTQ + communities [ 99 ]. This bias, which sidelines creators from diverse backgrounds, can limit viewers' exposure to content that might reflect their own identities (e.g. gender, sexuality, physical appearance, body functionality and cultural background). Thus there is a need to provide access to marginalised content creators in order to normalise variance and avoid confinement towards restrictive beauty standards. Recent work by Shrestha et al. [ 100 ], which co-designed digital interventions for body image with underrepresented populations, highlighted the vital importance that exposing individuals to diverse bodies can have in promoting positive body image. Their participants showed high enthusiasm for future interventions that supported increased diversity in social media (in terms of representation of diverse creators and non-appearance-related content). Biases can also be addressed through using diverse datasets during model training to minimise gender and cultural biases [ 101 ]. AI developers, in particular, should provide specific considerations to ensure their training data has fair and accurate representations of diversity. Cramer et al. [ 102 ] present helpful checklists for AI developers to help them think critically about diversity and representation by considering the purpose of the system, the output of the system and its impact on society and users before they begin developing it. However, a significant challenge in addressing these issues lies in the lack of algorithmic transparency on social media platforms. The opaque nature of these algorithms creates challenges for researchers and policymakers to assess what data is being used in training and how it influences content curation and recommendations [ 103 , 104 ]. This hinders efforts to identify and correct the biases. Despite this, it remains essential for developers and platforms to be mindful of these concerns and proactively adopt inclusive practices such as sociotechnical transparency frameworks [ 103 ] to create socially responsible AI systems. Recent work by Soubutts et al. [ 105 ] on digital mental health service design with culturally diverse young people presented a call to action for researchers to actively consider the engagement of culturally diverse people in technology design. During technological development, it is important to ensure that solutions resonate with individuals' needs, requirements and experiences. This is even more important in body image and ED, with significant gender and cultural biases in its literature base [ 100 , 106 ]. Understanding the context of harmful content Our findings indicated there is a need to create a robust set of rules for what is considered harmful and safe for people at-risk or with ED. This classification can be utilised to train algorithms to perform better content moderation, moving the field away from a reliance on hashtags and keywords. Evading hashtag moderation by using mutated hashtags, avoiding hashtags altogether, or hijacking and using hashtags meant for different content (e.g. body positive social media content) has been a go-to method for specific content creators and advertisers to keep their harmful content online [ 20 ]. However, before we can delve fully into the classification of these content, it is essential to understand the context surrounding certain content to ascertain whether it is harmful or not. Our findings showed that the intent of certain content is a key factor in determining its level of potential harm. Prior research has explored the use of AI technologies to generate captions and descriptions for image and video content by understanding and exploring the content's context [ 107 , 108 ]. This research can be leveraged in future work surrounding image-based social media classification by bolstering this generic contextual understanding with rules for classifying harmful content. For example, model alignment (a process of embedding human values into AI models and manipulating its output to align with human goals and rules) [ 109 ] within generative AI systems that can provide textual descriptions of videos and images (e.g. videoLlama, chatGPT). The implementation of highly robust methodologies used in healthcare research, such as the Delphi technique [ 110 ], which is a systematic process for gathering consensus, could be used in future research to drive the development of rules that are highly regarded as relevant and necessary for future AI systems aiming to classify harmful content. We propose that a useful direction for future research would be to develop AI systems that could be used by content creators. Rather than focusing solely on moderating harmful content after it appears on social media, it would be more effective and practical to prevent such content from being created and uploaded in the first place. In line with this preventative strategy, the Butterfly Foundation, in partnership with Instagram, has launched a social media series featuring Australian content creators who share their experiences with mindful content creation to avoid unintentional impact on body image perceptions, with a focus on promoting the well-being of their audiences [ 111 ]. This highlights the importance of educating content creators to be more aware and mindful of how their content can influence viewers, fostering a more responsible and supportive digital environment. Limitations ​​Despite recruitment efforts to recruit participants from diverse backgrounds for expert interviews and focus groups with individuals with lived experience, most of our participants were White and female. In addition, the majority of our lived experience participants had experiences of anorexia, meaning that the experiences of participants with bulimia, binge eating disorder and other ED classifications were not fully represented in our sample. While our research had a diversity level in its sample, our numbers were small. While a challenging task, it is vital that future research attempts to amplify the voices of these traditionally underrepresented voices in ED research. Approaching multiple lived experience networks, or recruiting directly from formal ED services, particularly those that focus on more marginalised experiences of ED, might aid more diversity in recruitment samples in the future. We did not collect data on participants' sexual orientation in this study. However, this could be considered in future research to provide a more comprehensive understanding. Conclusions Our paper reports on the range of contexts of potentially harmful content in social media for individuals at-risk of, or experiencing ED. The understanding of these contexts aims to empirically inform the future classification efforts of such content, and the role of technology in supporting this process. We have presented unique insights into the nuances and subjectivity of such content, and the factors that could tip the balance between harmful, ambiguous and safe content. This paper also presents potential technological solutions for future research to mitigate harmful social media content. However, challenges remain in developing technological solutions that safeguard against harmful content, while maintaining balance with 'freedom of speech' and avoiding censorship. We also emphasise the need to enhance the diverse representations in social media, by focusing on the promotion of non-appearance-related and diversified content regarding body shape, size, gender, body functionality and cultural backgrounds. Finally, we have provided a set of recommendations for the design of future technological solutions to reduce the negative impact of social media by supporting different approaches to content moderation. Abbreviations ED Eating disorders AI Artificial intelligence Declarations Ethics approval and consent to participate This study was approved by Monash University Human Research Ethics Committee, Monash University under the approval number ID- 41131. All participants provided informed consent prior to the participation. Consent for publication The participants provided informed consent for the publication of research data and findings, with the understanding that no identifying information would be disclosed in any publications. Availability of data and materials The data collected and analysed during the study are not publicly available due to the participant confidentiality requirements outlined in the ethics approval granted by the Monash University Human Research Ethics Committee. Competing interests The authors declare that they have no competing interests. Funding declaration No funding to be declared. Authors' contributions PS led the original draft, methodology, data collection, data analysis, and conceptualisation. JX, PDH, and RM contributed to the writing through review and editing and were involved in supervision and conceptualisation. MB supported the work through review and editing and supervision, while SG contributed to writing (review and editing) and provided support and resources for data collection. RM also played a key role in data analysis. References van Hoeken D, Hoek HW. Review of the burden of eating disorders: mortality, disability, costs, quality of life, and family burden. Curr Opin Psychiatry LWW; 2020;33(6):521–527. Gorrell S, Murray SB. Eating Disorders in Males. Child Adolesc Psychiatr Clin N Am 2019;28(4):641–651. doi: https://doi.org/10.1016/j.chc.2019.05.012 Mallaram GK, Sharma P, Kattula D, Singh S, Pavuluru P. Body image perception, eating disorder behavior, self-esteem and quality of life: a cross-sectional study among female medical students. J Eat Disord Springer; 2023;11(1):225. Castellano S, Rizzotto A, Neri S, Currenti W, Guerrera CS, Pirrone C, Coco M, Corrado DD. The Relationship between Body Dissatisfaction and Eating Disorder Symptoms in Young Women Aspiring Fashion Models: The Mediating Role of Stress. Eur J Investig Health Psychol Educ 2021;11(2):607–615. doi: https://doi.org/10.3390/ejihpe11020043 Rodgers RF, Melioli T. The relationship between body image concerns, eating disorders and internet use, part I: A review of empirical support. Adolesc Res Rev Springer; 2016;1:95–119. Leonard KC. The Impact of Social Media Body Challenges on Youths’ Body Image [Master’s Thesis]. North Dakota State University; 2020. Han SH, Safeek R, Ockerman K, Trieu N, Mars P, Klenke A, Furnas H, Sorice-Virk S. Public Interest in the Off-Label Use of Glucagon-like Peptide 1 Agonists (Ozempic) for Cosmetic Weight Loss: A Google Trends Analysis. Aesthet Surg J 2023 Dec 14;44(1):60–67. doi: 10.1093/asj/sjad211 Griffiths S, Castle D, Cunningham M, Murray SB, Bastian B, Barlow FK. How does exposure to thinspiration and fitspiration relate to symptom severity among individuals with eating disorders? Evaluation of a proposed model. Body Image Elsevier; 2018;27:187–195. Mahon C, Hevey D. Processing body image on social media: Gender differences in adolescent boys’ and girls’ agency and active coping. Front Psychol Frontiers Media SA; 2021;12:626763. Moorman EL, Warnick JL, Acharya R, Janicke DM. The use of internet sources for nutritional information is linked to weight perception and disordered eating in young adolescents. Appetite Elsevier; 2020;154. Sanzari CM, Gorrell S, Anderson LM, Reilly EE, Niemiec MA, Orloff NC, Anderson DA, Hormes JM. The impact of social media use on body image and disordered eating behaviors: Content matters more than duration of exposure. Eat Behav Elsevier; 2023;49:101722. Wu Y, Kemps E, Prichard I. Digging into digital buffets: A systematic review of eating-related social media content and its relationship with body image and eating behaviours. Body Image Elsevier; 2024;48:101650. Spence R, Bifulco A, Bradbury P, Martellozzo E, DeMarco J. The psychological impacts of content moderation on content moderators: A qualitative study. Cyberpsychology J Psychosoc Res Cyberspace 2023 Sep 18;17(4). doi: 10.5817/CP2023-4-8 Gillespie T. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press; 2019. doi: 10.12987/9780300235029ISBN:978-0-300-23502-9 How We’re Supporting People Affected by Eating Disorders and Negative Body Image. Meta; 2021. doi: https://about.fb.com/news/2021/02/supporting-people-affected-by-eating-disorders-and-negative-body-image/ Help Center. Instagram; 2024. doi: https://help.instagram.com/252214974954612 Safety Center. TikTok; 2024. doi: https://www.tiktok.com/safety/en/eating-disorder Peebles R, Wilson JL, Litt IF, Hardy KK, Lock JD, Mann JR, Borzekowski DL. Disordered Eating in a Digital Age: Eating Behaviors, Health, and Quality of Life in Users of Websites With Pro-Eating Disorder Content. J Med INTERNET Res. ‘Thinstagram’: Instagram’s algorithm fuels eating disorder epidemic. Tech Transparency Project; 2021. doi: https://www.techtransparencyproject.org/articles/thinstagram-instagrams-algorithm-fuels-eating-disorder-epidemic Gerrard Y. Beyond the hashtag: Circumventing content moderation on social media. New Media Soc SAGE Publications Sage UK: London, England; 2018;20(12):4492–4511. Drivas M, Reed OS, Berndt-Goke M. #WhatIEatInADay: The effects of viewing food diary TikTok videos on young adults’ body image and intent to diet. Body Image 2024 Jun;49:101712. doi: 10.1016/j.bodyim.2024.101712 Davis HA, Patarinski AGG, Kosmas JA, Roske C, Kells M. Does watching short-form #WhatIEatInADay videos impact eating disorder cognitions and urges to engage in eating disorder behaviors? An experimental investigation of TikTok. Appetite 2025 Jul;211:108002. doi: 10.1016/j.appet.2025.108002 Gagrčin E. Your social ties, your personal public sphere, your responsibility: How users construe a sense of personal responsibility for intervention against uncivil comments on Facebook. New Media Soc 2024 Aug;26(8):4299–4316. doi: 10.1177/14614448221117499 Kaňková J, Binder A, Matthes J. Helpful or harmful? Navigating the impact of social media influencers’ health advice: insights from health expert content creators. BMC Public Health 2024 Dec 18;24(1):3511. doi: 10.1186/s12889-024-21095-3 Haime Z, Kennedy L, Grace L, Biddle L. Experiences of Moderation, Moderators, and Moderating by Online Users Who Engage with Self-Harm and Suicide Content. Digit Soc 2025 Apr;4(1):8. doi: 10.1007/s44206-025-00166-x Gongane VU, Munot MV, Anuse AD. Detection and moderation of detrimental content on social media platforms: current status and future directions. Soc Netw Anal Min 2022 Dec;12(1):129. doi: 10.1007/s13278-022-00951-3 Yan H, Fitzsimmons-Craft EE, Goodman M, Krauss M, Das S, Cavazos-Rehg P. Automatic detection of eating disorder-related social media posts that could benefit from a mental health intervention. Int J Eat Disord Wiley Online Library; 2019;52(10):1150–1156. Ramírez-Cifuentes D, Mayans M, Freire A. Early risk detection of anorexia on social media. Internet Sci 5th Int Conf INSCI 2018 St Petersburg Russ Oct 24–26 2018 Proc 5 Springer; 2018. p. 3–14. Abuhassan M, Anwar T, Liu C, Jarman HK, Fuller-Tyszkiewicz M. EDNet: Attention-Based Multimodal Representation for Classification of Twitter Users Related to Eating Disorders. Proc ACM Web Conf 2023 2023. p. 4065–4074. Park J, Gracie J, Alsoubai A, Stringhini G, Singh V, Wisniewski P. Towards Automated Detection of Risky Images Shared by Youth on Social Media. Companion Proc ACM Web Conf 2023 Austin TX USA: ACM; 2023. p. 1348–1357. doi: 10.1145/3543873.3587607 Dey AK. Understanding and Using Context. Pers Ubiquitous Comput 2001; How does Facebook use artificial intelligence to moderate content? Meta; 2024. doi: https://www.facebook.com/help/1584908458516247 Setting the rules for a safe TikTok experience. TikTok; 2024. doi: https://www.tiktok.com/transparency/en-gb/content-moderation/ How Instagram uses artificial intelligence to moderate content. Instagram; 2024. doi: https://help.instagram.com/423837189385631 Shirky C. The political power of social media: Technology, the public sphere, and political change. Foreign Aff JSTOR; 2011;28–41. Facebook Community Standards. Meta; 2024. doi: https://transparency.meta.com/en-gb/policies/community-standards/?source=https%3A%2F%2Fwww.facebook.com%2Fcommunitystandards Hall R, Wilmot C. Meta faces Ghana lawsuits over impact of extreme content on moderators. The Guardian 2025 Apr 27; Available from: https://www.theguardian.com/technology/2025/apr/27/meta-faces-ghana-lawsuits-over-impact-of-extreme-content-on-moderators Card C. How Facebook AI Helps Suicide Prevention. Meta; Meta M. Suicide prevention. Available from: https://about.meta.com/actions/safety/topics/wellbeing/suicideprevention Help Centre. If you see a post on Facebook that suggests someone has an eating disorder. Available from: https://www.facebook.com/help/230070543674737/ Eating-disorder patients confront Meta bosses at federal parliament as research highlights TikTok’s negative influence. ABC News; 2023. doi: https://www.abc.net.au/news/2023-09-14/eating-disorder-patients-to-confront-meta-bosses-at-parliament/102847320 Griffiths S, Harris EA, Whitehead G, Angelopoulos F, Stone B, Grey W, Dennis S. Does TikTok contribute to eating disorders? A comparison of the TikTok algorithms belonging to individuals with eating disorders versus healthy controls. Body Image 2024 Dec;51:101807. doi: 10.1016/j.bodyim.2024.101807 YouTube’s Anorexia Algorithm How YouTube recommends eating disorders videos to young girls. Center for Countering Digital Hate; 2024. Available from: https://counterhate.com/wp-content/uploads/2024/12/CCDH.YoutubeED.Nov24.Report_FINAL.pdf Rodgers RF, McLean SA, Paxton SJ. Enhancing understanding of social media literacy to better inform prevention of body image and eating disorders. Eat Disord Taylor & Francis; 2024;1–19. Jhaver S, Zhang AQ, Chen Q, Natarajan N, Wang R, Zhang A. Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor. arXiv; 2023. doi: 10.48550/arXiv.2305.10374 Holland G, Tiggemann M. A systematic review of the impact of the use of social networking sites on body image and disordered eating outcomes. Body Image Elsevier; 2016;17:100–110. Jiotsa B, Naccache B, Duval M, Rocher B, Grall-Bronnec M. Social media use and body image disorders: Association between frequency of comparing one’s own physical appearance to that of people being followed on social media and body dissatisfaction and drive for thinness. Int J Environ Res Public Health MDPI; 2021;18(6):2880. Casale S, Gemelli G, Calosi C, Giangrasso B, Fioravanti G. Multiple exposure to appearance-focused real accounts on Instagram: Effects on body image among both genders. Curr Psychol Springer; 2021;40:2877–2886. Pedalino F, Camerini A-L. Instagram use and body dissatisfaction: The mediating role of upward social comparison with peers and influencers among young females. Int J Environ Res Public Health MDPI; 2022;19(3):1543. McLean SA, Wertheim EH, Masters J, Paxton SJ. A pilot evaluation of a social media literacy intervention to reduce risk factors for eating disorders. Int J Eat Disord Wiley Online Library; 2017;50(7):847–851. Rodgers RF, Wertheim EH, Paxton SJ, Tylka TL, Harriger JA. # Bopo: Enhancing body image through body positive social media-evidence to date and research directions. Body Image Elsevier; 2022;41:367–374. Tylka TL, Wood-Barcalow NL. What is and what is not positive body image? Conceptual foundations and construct definition. Body Image Elsevier; 2015;14:118–129. McLean SA, Wertheim EH, Marques MD, Paxton SJ. Dismantling prevention: Comparison of outcomes following media literacy and appearance comparison modules in a randomised controlled trial. J Health Psychol Sage Publications Sage UK: London, England; 2019;24(6):761–776. Social media literacy. SMRI; 2024. doi: https://www.smri.world/dictionary/social-media-literacy Wood-Barcalow NL, Tylka TL, Augustus-Horvath CL. “But I like my body”: Positive body image characteristics and a holistic model for young-adult women. Body Image Elsevier; 2010;7(2):106–116. Andrew R, Tiggemann M, Clark L. Predictors and health-related outcomes of positive body image in adolescent girls: A prospective study. Dev Psychol American Psychological Association; 2016;52(3):463. Cohen R, Irwin L, Newton-John T, Slater A. # bodypositivity: A content analysis of body positive accounts on Instagram. Body Image Elsevier; 2019;29:47–57. Webb JB, Vinoski ER, Bonar AS, Davies AE, Etzel L. Fat is fashionable and fit: A comparative content analysis of Fatspiration and Health at Every Size® Instagram images. Body Image Elsevier; 2017;22:53–64. Brino KAS, Derouin AL, Silva SG. Problematic internet use in adolescents and implementation of a social media hygiene protocol. J Pediatr Nurs Elsevier; 2022;63:84–89. Haime Z, Kennedy L, Grace L, Cohen R, Derges J, Biddle L. The Journey of Engaging With Web-Based Self-Harm and Suicide Content: Longitudinal Qualitative Study. JMIR Infodemiology 2024 Mar 28;4:e47699. doi: 10.2196/47699 Carey M. The Affects Social Media has on People with Eating Disorders. Ir Assoc Couns Psychother 2015;15(4). Khurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl Springer; 2023;82(3):3713–3744. Benítez-Andrades JA, Alija-Pérez JM, García-Rodríguez I, Benavides C, Alaiz-Moretón H, Vargas RP, García-Ordás MT. BERT Model-Based Approach For Detecting Categories of Tweets in the Field of Eating Disorders (ED). 2021 IEEE 34th Int Symp Comput-Based Med Syst CBMS IEEE; 2021. p. 586–590. Feldman J. Classifying Pro-Eating Disorder Images on Twitter Using Deep Learning. Intersect Stanf J Sci Technol Soc 2023;16(2). Khan AA, Laghari AA, Awan SA. Machine learning in computer vision: a review. EAI Endorsed Trans Scalable Inf Syst 2021;8(32):e4–e4. Chancellor S, Pater JA, Clear T, Gilbert E, De Choudhury M. #thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities. Proc 19th ACM Conf Comput-Support Coop Work Soc Comput 2016. p. 1201–1213. Llansó E, van Hoboken J, Leerssen P, Harambam J. Artificial Intelligence, Content Moderation, and Freedom of Expression. Hosseini H, Kannan S, Zhang B, Poovendran R. Deceiving Google’s Perspective API Built for Detecting Toxic Comments. arXiv; 2017. doi: 10.48550/arXiv.1702.08138 Lees A, Tran VQ, Tay Y, Sorensen J, Gupta J, Metzler D, Vasserman L. A New Generation of Perspective API: Efficient Multilingual Character-level Transformers. arXiv; 2022. doi: 10.48550/arXiv.2202.11176 Cavus N, Goksu M, Oktekin B. Real-time fake news detection in online social networks: FANDC Cloud-based system. Sci Rep 2024 Oct 29;14(1):25954. doi: 10.1038/s41598-024-76102-9 Maham S, Tariq A, Khan MUG, Alamri FS, Rehman A, Saba T. ANN: adversarial news net for robust fake news classification. Sci Rep 2024 Apr 3;14(1):7897. doi: 10.1038/s41598-024-56567-4 Zhang C, Gupta A, Qin X, Zhou Y. A computational approach for real-time detection of fake news. Expert Syst Appl 2023 Jul;221:119656. doi: 10.1016/j.eswa.2023.119656 Khan SA, Sheikhi G, Opdahl AL, Rabbi F, Stoppel S, Trattner C, Dang-Nguyen D-T. Visual User-Generated Content Verification in Journalism: An Overview. IEEE Access 2023;11:6748–6769. doi: 10.1109/ACCESS.2023.3236993 Urbani S. Verifying Online Information. First Draft; Available from: https://firstdraftnews.org/long-form-article/verifying-online-information/ Bacha J, Ullah F, Khan J, Sardar AW, Lee S. A Deep Learning-Based Framework for Offensive Text Detection in Unstructured Data for Heterogeneous Social Media. IEEE Access 2023;11:124484–124498. doi: 10.1109/ACCESS.2023.3330081 Suicide, self-injury and eating disorders. Meta; Available from: https://transparency.meta.com/en-gb/policies/community-standards/suicide-self-injury/ Arora A, Nakov P, Hardalov M, Sarwar SM, Nayak V, Dinkov Y, Zlatkova D, Dent K, Bhatawdekar A, Bouchard G, Augenstein I. Detecting Harmful Content on Online Platforms: What Platforms Need vs. Where Research Efforts Go. ACM Comput Surv 2024 Mar 31;56(3):1–17. doi: 10.1145/3603399 Braun V, Clarke V. Successful Qualitative Research a practical guide for beginners. Proposed Social Media Ban for Under-16s in Australia. Aust Hum Rights Comm 2024 Nov 21; Available from: https://humanrights.gov.au/about/news/proposed-social-media-ban-under-16s-australia McNaney R, Vines J, Mercer J, Mexter L, Welsh D, Young T. DemYouth: Co-designing and enacting tools to support young people’s engagement with people with dementia. Proc 2017 CHI Conf Hum Factors Comput Syst New York, NY, USA: ACM; 2017. p. 1313–1325. doi: https://doi.org/10.1145/3025453.3025558 Briggs P, Blythe M, Vines J, Lindsay S, Dunphy P, Nicholson J, Green D, Kitson J, Monk A, Olivier P. Invisible design: exploring insights and ideas through ambiguous film scenarios. Proc Des Interact Syst Conf Newcastle Upon Tyne United Kingdom: ACM; 2012. p. 534–543. doi: 10.1145/2317956.2318036 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77–101. Carter N, Bryant-Lukosius D, DiCenso A, Blythe J, Neville AJ. The Use of Triangulation in Qualitative Research. Oncol Nurs Forum 2014 Sep 1;41(5):545–547. doi: 10.1188/14.ONF.545-547 Michael Nnaemeka Ajemba, Ebube Chinwe Arene. Possible advantages that may be enhanced with the adoption of research triangulation or mixed methodology. Magna Sci Adv Res Rev 2022 Sep 30;6(1):058–061. doi: 10.30574/msarr.2022.6.1.0066 Hockin-Boyers H, Pope S, Jamie K. Digital pruning: Agency and social media use as a personal political project among female weightlifters in recovery from eating disorders. New Media Soc SAGE Publications Sage UK: London, England; 2021;23(8):2345–2366. Khursheed A. Role of Media Literacy in Spreading Body Positivity and in Preventing Eating Disorder. Int J Indian Psychȯlogy 2023;11(1). Fardouly J, Slater A, Parnell J, Diedrichs PC. Can following body positive or appearance neutral Facebook pages improve young women’s body image and mood? Testing novel social media micro-interventions. Body Image Elsevier; 2023;44:136–147. Lewis-Smith H, Garbett KM, Chaudhry A, Dhillon M, Shroff H, White P, Diedrichs PC. Evaluating a body image school-based intervention in India: A randomized controlled trial. Body Image Elsevier; 2023;44:148–156. Ghaznavi J, Taylor LD. Bones, body parts, and sex appeal: An analysis of# thinspiration images on popular social media. Body Image Elsevier; 2015;14:54–61. Tiggemann M, Churches O, Mitchell L, Brown Z. Tweeting weight loss: A comparison of# thinspiration and# fitspiration communities on Twitter. Body Image Elsevier; 2018;25:133–138. Vega J, Bell BT, Taylor C, Xie J, Ng H, Honary M, McNaney R. Detecting mental health behaviors using mobile interactions: Exploratory study focusing on binge eating. JMIR Ment Health JMIR Publications Toronto, Canada; 2022;9(4):e32146. Kozyreva A, Herzog SM, Lewandowsky S, Hertwig R, Lorenz-Spreen P, Leiser M, Reifler J. Resolving content moderation dilemmas between free speech and harmful misinformation. Proc Natl Acad Sci National Acad Sciences; 2023;120(7):e2210666120. Herrick SSC, Hallward L, Duncan LR. “This is just how I cope”: An inductive thematic analysis of eating disorder recovery content created and shared on TikTok using # EDrecovery. Int J Eat Disord 2021 Apr;54(4):516–526. doi: 10.1002/eat.23463 Not just algorithms: assuring user safety online with systemic regulatory frameworks. Reset Australia; 2024. doi: https://apo.org.au/node/326122 Au ES, Cosh SM. Social media and eating disorder recovery: An exploration of Instagram recovery community users and their reasons for engagement. Eat Behav Elsevier; 2022;46:101651. Rong EZ, Zhou MM, Lu Z, Fan M. “It Feels Like Being Locked in A Cage”: Understanding Blind or Low Vision Streamers’ Perceptions of Content Curation Algorithms. Proc 2022 ACM Des Interact Syst Conf 2022. p. 571–585. Biddle S, Riberio PV, Dias T. Invisible Censorship: TikTok Told Moderators to Suppress Posts by “Ugly” People and the Poor to Attract New Users. The Intercept; 2020. doi: https://theintercept.com/2020/03/16/tiktok-app-moderatorsusers-discrimination/ Karizat N, Delmonaco D, Eslami M, Andalibi N. Algorithmic folk theories and identity: How TikTok users co-produce Knowledge of identity and engage in algorithmic resistance. Proc ACM Hum-Comput Interact ACM New York, NY, USA; 2021;5(CSCW2):1–44. Simpson E, Semaan B. For you, or for" you"? Everyday LGBTQ+ encounters with TikTok. Proc ACM Hum-Comput Interact ACM New York, NY, USA; 2021;4(CSCW3):1–34. Shrestha P, Delir Haghighi P, Sharp G, Xie J, McNaney R. One Size (Doesn’t) Fit All: Exploring design considerations for digital body dissatisfaction interventions with underrepresented populations. Proc 2024 ACM Des Interact Syst Conf 2024. p. 3534–3550. Wang T, Zhao J, Yatskar M, Chang K-W, Ordonez V. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. Proc IEEECVF Int Conf Comput Vis 2019. p. 5310–5319. Cramer H, Garcia-Gathright J, Springer A, Reddy S. Assessing and addressing algorithmic bias in practice. Interactions ACM New York, NY, USA; 2018;25(6):58–63. Gausen A, Guo C, Luk W. An approach to sociotechnical transparency of social media algorithms using agent-based modelling. AI Ethics 2025 Apr;5(2):1827–1845. doi: 10.1007/s43681-024-00527-1 Exploring social media users’ experiences with algorithmic transparency cues. New Media. Soubutts E, Shrestha P, Davidson BI, Qu C, Mindel C, Sefi A, Marshall P, McNaney R. Challenges and Opportunities for the Design of Inclusive Digital Mental Health Tools: Understanding Culturally Diverse Young People’s Experiences. 2024. p. 1–16. Borowsky HM, Eisenberg ME, Bucchianeri MM, Piran N, Neumark-Sztainer D. Feminist identity, body image, and disordered eating. Eat Disord Taylor & Francis; 2016;24(4):297–311. Gao L, Guo Z, Zhang H, Xu X, Shen HT. Video captioning with attention-based LSTM and semantic consistency. IEEE Trans Multimed IEEE; 2017;19(9):2045–2055. Han T, Bain M, Nagrani A, Varol G, Xie W, Zisserman A. AutoAD: Movie description in context. Proc IEEECVF Conf Comput Vis Pattern Recognit 2023. p. 18930–18940. Shen T, Jin R, Huang Y, Liu C, Dong W, Guo Z, Wu X, Liu Y, Xiong D. Large language model alignment: A survey. ArXiv Prepr ArXiv230915025 2023; Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol Baishideng Publishing Group Inc; 2021;11(4):116. Pause before you post: Social media ‘appearance-led’ content impacting Australians’ body image. Butterfly foundation; Available from: https://butterfly.org.au/news/pause-before-you-post-social-media-appearance-led-content-impacting-australians-body-image/ Additional Declarations No competing interests reported. Supplementary Files AppendixA.docx AppendixB.docx Cite Share Download PDF Status: Published Journal Publication published 21 Jan, 2026 Read the published version in Journal of Eating Disorders → Version 1 posted Editorial decision: Revision requested 17 Nov, 2025 Reviews received at journal 16 Nov, 2025 Reviewers agreed at journal 30 Oct, 2025 Reviews received at journal 25 Jul, 2025 Reviewers agreed at journal 25 Jul, 2025 Reviewers invited by journal 25 Jul, 2025 Editor assigned by journal 18 Jul, 2025 Submission checks completed at journal 18 Jul, 2025 First submitted to journal 16 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7136130","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":503189795,"identity":"15615878-e572-4626-b747-3fb23c70dd71","order_by":0,"name":"Pranita Shrestha","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABHklEQVRIie2QMUvDQBTHXykky0nWC4fNV7gSSBGq/SpPAueSwcnFwU51aXUt5EsUhMyBB+2SWQLJ0CyZurhIChlMpYLYBB0d7jfcPd7x4//eAWg0/xU8HPyzjMGCY3Hsdyj4pWAM9vQvCnxXZPyLMhJUbovqCpxwURT7KnfcV1pvqzoHywxkm3LxpEYS0QeZb1yXYTmMMnUzXMxKsOe7VkUm4HHEPkiuDAFIvSgLPH42JZBpe4pMzPdGeQBnqcx9hTR5CQPPrmuCSafCDikEkCoDGNL1SgSeYEaTwjt2mbM7jmrDZKr6ginyl5lS4nxGjCflbeuPMTOyq/H9oBms91aN6fI59Nf2rqaB9eivWgc73uzk5bTzQ9FoNBpNNx+tMmNkECUFXAAAAABJRU5ErkJggg==","orcid":"","institution":"Monash University","correspondingAuthor":true,"prefix":"","firstName":"Pranita","middleName":"","lastName":"Shrestha","suffix":""},{"id":503189797,"identity":"081a949a-62e6-4d69-ae84-67ba99924bc0","order_by":1,"name":"Jue Xie","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Jue","middleName":"","lastName":"Xie","suffix":""},{"id":503189799,"identity":"26cddd82-7d36-481f-88d6-930bfb76da6d","order_by":2,"name":"Pari Delir Haghighi","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Pari","middleName":"Delir","lastName":"Haghighi","suffix":""},{"id":503189801,"identity":"ca4773a6-3797-4357-a856-fc9ae9f9adee","order_by":3,"name":"Michelle L. Byrne","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Michelle","middleName":"L.","lastName":"Byrne","suffix":""},{"id":503189803,"identity":"01754ccd-c2c4-408e-ad25-914a2d9db278","order_by":4,"name":"Scott Griffiths","email":"","orcid":"","institution":"University of Melbourne","correspondingAuthor":false,"prefix":"","firstName":"Scott","middleName":"","lastName":"Griffiths","suffix":""},{"id":503189805,"identity":"2364322c-3b05-4cfd-8480-1cbbf370d3bc","order_by":5,"name":"Roisin McNaney","email":"","orcid":"","institution":"University of Melbourne","correspondingAuthor":false,"prefix":"","firstName":"Roisin","middleName":"","lastName":"McNaney","suffix":""}],"badges":[],"createdAt":"2025-07-16 05:53:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7136130/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7136130/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s40337-025-01504-7","type":"published","date":"2026-01-21T15:58:20+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":90285311,"identity":"6a322c18-638b-4961-b713-9b1c2345dc4f","added_by":"auto","created_at":"2025-09-01 06:00:12","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":217672,"visible":true,"origin":"","legend":"\u003cp\u003eFirst activity of focus group focusing on the role of social media in different stages of eating disorder\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7136130/v1/0e7fdbe11a38768502c13fff.png"},{"id":101151756,"identity":"b03797fe-6b0f-4b75-b05f-c19fd72c8848","added_by":"auto","created_at":"2026-01-26 16:04:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1478856,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7136130/v1/12eae315-c074-4766-98ae-14eb3807bd4b.pdf"},{"id":90284233,"identity":"fffcbb97-9dd3-4691-8b00-b588f08432d0","added_by":"auto","created_at":"2025-09-01 05:52:12","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":11100,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixA.docx","url":"https://assets-eu.researchsquare.com/files/rs-7136130/v1/9c339ce79c12c46ec934bdfd.docx"},{"id":90285312,"identity":"f58b8003-3a31-4feb-bb15-2f4b78c05ca4","added_by":"auto","created_at":"2025-09-01 06:00:12","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":17098,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixB.docx","url":"https://assets-eu.researchsquare.com/files/rs-7136130/v1/3ec16243baba05095c92328e.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Thigh Gaps and Filtered Snaps: A Qualitative Study Exploring Opportunities to Mitigate Social Media Harm Through Content Moderation for People with Eating Disorders","fulltext":[{"header":"Plain English Summary","content":"\u003cp\u003eConstant exposure to idealised beauty standards online can lead to negative body image, unhealthy behaviours and even eating disorders. To help reduce these harms, we need better ways to identify and moderate harmful content. However, first, we must understand why and how this content causes harm. We consulted n=30 participants, including interviews with n=12 professionals in ED support and research, and focus groups with n=18 individuals with lived experience of eating disorders. Six major themes emerged: the role of context in assessing harmful content, social media's contribution to an ED \"echo chamber,\" challenges in content moderation, the need for safer user experiences, the importance of diverse representation and the potential for technology to help mitigate harm. From this analysis, we developed eight distinct categories of harmful social media content related to body image and eating disorders. These categories help clarify the various ways individuals may be negatively affected by harmful content and provide structure for future solutions. Participants highlighted the powerful influence of algorithms in promoting harmful content and called for shared responsibility among users, content creators, platforms, and policymakers. The findings offer guidance for designing technologies that mitigate social media harm.\u003c/p\u003e"},{"header":"Background","content":"\u003cp\u003eEach year, approximately 3.3\u0026nbsp;million individuals are affected by the physical and mental impact of eating disorders [\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e]. Eating disorders (ED) are severe mental health conditions characterised by unhealthy behaviours towards eating, food, and exercise, such as extreme food restriction or binge eating, relentless fixation on body shape and weight, and engagement with harmful compensatory actions like purging after eating (e.g. vomiting, overexercising, laxative use) [\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e]. They are closely connected with body image concerns. Body image refers to the thoughts, perceptions, feelings, and emotions about one\u0026apos;s body regarding appearance, shape, size, and other physical attributes [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e]. Individuals with negative body image can often develop disordered eating behaviours (chronic dieting, purging calories through dieting, skipping meals, or obsessive calorie counting), which in turn can lead to the development of ED [\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eSocial media platforms amplify content that promotes what might be considered \u0026apos;ideal\u0026apos; beauty standards regarding physical attractiveness and desirability, typically shaped by media, influencers and cultural norms [\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e]. The constant exposure of this kind of content can influence individuals\u0026apos; behaviour, to emulate unrealistic \u0026apos;ideals.\u0026apos; A prime example is the rise of online \u0026apos;challenges\u0026apos; such as the thigh gap challenge, where women strive to achieve a noticeable gap between their inner thighs when standing with their feet together [\u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e]. Aside from this, popularisation of certain drugs for weight loss, such as Ozempic by celebrities and content creators, has also been rampant in social media [\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e]. In addition, broader content within the health and fitness category may promote specific ideals of muscularity or fitness to aspire to, both of which can lead to the development of unhealthy eating and exercise behaviours in an attempt to achieve this [\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e8\u003c/span\u003e]. Research has additionally shown that adolescents frequently use social media as a primary source of information about diet, nutrition and fitness, often from creators who may not be credible professional sources, or who might be promoting extreme methods framed as fad diets (e.g. eating an unbalanced diet by cutting out food groups such as carbohydrates, demonisation of sugar, fasting) [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e]. Numerous research studies have found a clear connection between the type of content users view on social media, body dissatisfaction and increased disordered eating behaviours [\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eWith the rise of user-generated social media content, content moderation has become crucial to creating a safer online environment. Content moderation is monitoring and identifying harmful content that goes against legal, ethical and community standards and taking necessary steps towards removing such content [\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e]. Several popular platforms (e.g. Facebook [\u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e], Instagram [\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e], TikTok [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e]) additionally state that they remove pro-ED content in their guidelines. Pro-ED content is content that promotes ED as a \u0026apos;lifestyle choice\u0026apos; rather than a severe mental condition, and reinforces and offers encouragement to partake in disordered eating behaviours [\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e]. The platforms heavily rely on hashtags and keywords for conducting this moderation of pro-ED content (e.g. pro-ana, ana - short form for anorexia nervosa, mia - short form for bulimia nervosa, anorexia, bulimia, thinspiration). Users who search for these terms are immediately redirected to the respective country\u0026apos;s helpline services [\u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e]. However, research has found that users have developed methods for bypassing hashtag moderation, by avoiding hashtags entirely or using altered versions of hashtags that can easily evade the platform\u0026apos;s moderation systems [\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e]. Self-moderation is another form of content moderation where the users themselves proactively moderate and regulate the kind of content they interact with. However, to achieve this, it is important to equip the individuals with the concept of what is harmful to them and how to navigate the social media space by putting their well-being at the forefront.\u003c/p\u003e\n\u003cp\u003eThe transient and ever-changing nature of social media content creates further difficulty for content moderation. For example, the emerging trend of \u0026quot;What I eat in a day\u0026quot; videos, where individuals provide information about their daily meals, calorie counts or portion sizes [\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e]. While this type of content may seem harmless or even informative and educational, it often falls into an \u0026quot;ambiguous\u0026quot; or \u0026quot;grey area\u0026quot; category of content depending on the context in which they are presented and consumed [\u003cspan class=\"CitationRef\"\u003e21\u003c/span\u003e]. For individuals who are at-risk of or experiencing ED, this kind of content can potentially lead to obsessive calorie tracking, food fixation and body dissatisfaction [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. Such hyperfixation on calorie and food can be damaging for vulnerable individuals especially when social media algorithms create a loop of such content to increase engagement. Thus the benefit and risk from such ambiguous content depends on the level of self-awareness and resilience in the individual. This highlights the importance of ensuring that content moderation efforts by individuals and platforms are dynamic and adaptable so that harmful trends can be detected in the early stages.\u003c/p\u003e\n\u003cp\u003eResearch has highlighted that creating a safe social media environment requires a collective and shared commitment from individuals, content creators, human moderators, platforms and governments [\u003cspan class=\"CitationRef\"\u003e23\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e]. Due to the ever-growing volume of social media content, such collaborative efforts from different stakeholders can be enhanced further through automated technological systems. These can go beyond those already in place by social media platforms so as to reduce the load (on individuals, content creators and human moderators) to make content moderation choices. In the ED space, studies have explored using machine learning techniques to identify ED-related Reddit posts [\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e]. In addition, Abuhassan et al. [\u003cspan class=\"CitationRef\"\u003e29\u003c/span\u003e] applied machine learning techniques to differentiate users at-risk of developing ED, from users who are merely engaging with ED-related content, based on their interactions on Twitter (now X). While much of the existing research has been concentrated on identifying and categorising content on text-based social media platforms, the growing influence and negative impact of image-centric platforms such as Instagram, TikTok and Snapchat highlight an urgent need to extend content identification and moderation to media beyond text (tweets, caption, hashtags, keywords). However, the level of subjectivity surrounding visual content (i.e. understanding what might be considered harmful and for whom) and the content\u0026apos;s dependence on contextual nuances, creates a challenge in developing approaches to automation for detecting potentially harmful content [\u003cspan class=\"CitationRef\"\u003e30\u003c/span\u003e]. Context can be defined as the essential background information that facilitates an understanding of the particular situation [\u003cspan class=\"CitationRef\"\u003e31\u003c/span\u003e]. Thus, it is important to understand the context of such content to understand possible underlying subjectivities and to design and develop technology to provide better content navigation and moderation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrior Work\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCurrent state of content moderation of social media\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSocial media platforms each have a version of automatic content moderation [\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e], to remove certain content that might be against their respective community guidelines. These are usually guided by legal factors [\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e]. For example, Meta\u0026apos;s community standards strictly mention the removal of content including child abuse, nudity, suicide and self-harm, explicit eating disorder (ED) content, and hate speech [\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e]. These platforms have AI-driven content moderation in place, which analyses the content (visual, audio) and attached textual data (keyword, hashtag, title, caption) to determine its safety [\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e]. If content is deemed unsafe, it is either entirely removed or its spread is limited (e.g. not shown to users under 18). Whilst this is a necessary first step in keeping social media communities safe, there is a distinct lack of transparency in the algorithms that make these decisions, and often only the most extreme content is flagged. In addition, there is a human cost associated with the development of these content moderation algorithms. Meta is currently facing a lawsuit involving human content moderators in Ghana who have reportedly suffered mental health issues after being exposed to disturbing and harmful content on social media, including extreme violence, murders, and child sexual abuse [\u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e]. The lawsuit attributes these conditions to a lack of adequate safeguards to protect the well-being of the workers. This case highlights the urgent need for better automated content moderation work across all categories of harmful content, including eating disorders where human moderators are exposed to content such as self-starvating, disordered eating behaviours and body shaming. Specifically for self-harm and suicide prevention, when Facebook\u0026apos;s AI algorithm flags a potential case, Facebook first provides a support option with resources. In extremely serious cases determined by Facebook\u0026apos;s Community Operations teams, Facebook directly contacts local authorities to perform wellness checks [\u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e39\u003c/span\u003e]. According to Meta\u0026apos;s official website [\u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e], in 2018 they performed over 1000 wellness checks with the support of first responders. For ED, Facebook encourages individuals to report posts that suggest someone is experiencing an ED so that Facebook can reach out to them [\u003cspan class=\"CitationRef\"\u003e40\u003c/span\u003e], however how this is practically implemented, and at what scale in the global reach that facebook has, is unknown.\u003c/p\u003e\n\u003cp\u003eDespite these efforts from social media platforms a significant amount of ED-related content is still highly prevalent [\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e]. In a study by Griffiths et al. [\u003cspan class=\"CitationRef\"\u003e42\u003c/span\u003e] the authors reported that the TikTok algorithm recommended 4343% more harmful ED content, 335% more dieting content and 142% more exercise content to people experiencing ED in comparison to people not experiencing ED. Similarly, a recent study by the Center for Countering Digital Hate \u0026quot;YouTube\u0026apos;s Anorexia Algorithm\u0026quot; [\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e] reported that amongst 1000 videos recommended by the Youtube algorithm to a hypothetical 13-year old girl (a profile created by researchers), 34.4% were considered as harmful eating disorder content (breaching YouTube\u0026apos;s safety policies) and 63.8% were videos related to weight loss or ED. As such, despite content moderation safeguards placed by certain platforms, ED-related content still reaches viewers in large amounts.\u003c/p\u003e\n\u003cp\u003eBesides automated content moderation, platforms offer reporting and flagging options to the users themselves. Once content has been flagged, human moderators from the specific platform will review the content and make a final decision regarding whether or not the content breaches community guidelines [\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e34\u003c/span\u003e]. Platforms also provide options to users to unfollow, mute and block certain content to enhance their ability to curate and moderate their feeds [\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e]. However, putting all the responsibility of moderation on the user themselves can be burdensome, as not all users have the same cognitive and critical thinking abilities to identify harmful content [\u003cspan class=\"CitationRef\"\u003e45\u003c/span\u003e]. As such, this requires a shared responsibility for moderation between users and platforms.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTechnology used for mitigating harm in online spaces\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNumerous research studies have used Natural Language Processing (NLP) (a branch of Artificial Intelligence (AI) that understands and interprets human language) [\u003cspan class=\"CitationRef\"\u003e63\u003c/span\u003e] to understand harmful content from textual data such as captions, hashtags and keywords to identify pro-ED content [\u003cspan class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e63\u003c/span\u003e]. Feldman et al. [\u003cspan class=\"CitationRef\"\u003e64\u003c/span\u003e] moved from just text-based analysis to incorporating both static image and text in social media content to classify harmful or safe content using Computer Vision (another branch of AI that understands and interprets visual information) [\u003cspan class=\"CitationRef\"\u003e65\u003c/span\u003e]. The study developed an image classifier to distinguish between harmful (promoting ED) and safe (not promoting ED) content based on visual cues. As the study suggests, it is important to consider the visual aspect of social media due to the increase in the popularity of visual-based social media platforms. While this research is promising, Chancellor et al. [\u003cspan class=\"CitationRef\"\u003e66\u003c/span\u003e] stressed the importance of combining automated content moderation with alternative intervention solutions, rather than simply implementing blanket bans on certain content types. The authors described that content creators find ways to circumvent content moderation strategies (e.g. by altering hashtags that might be flagged). Additionally, as social media transitions from static images to dynamic video content, understanding visual context and subtle nuances becomes increasingly important.\u003c/p\u003e\n\u003cp\u003eWith a growing scale of harmful content in online spaces, there has been an increase in the development of automated technologies to detect and analyse content regarding hate speech, terrorism and misinformation [\u003cspan class=\"CitationRef\"\u003e67\u003c/span\u003e]. Google Jigsaw developed \u0026apos;Perspective API\u0026apos; which used NLP to detect and manage toxic comments in online platforms [\u003cspan class=\"CitationRef\"\u003e68\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e69\u003c/span\u003e]. The API categorised toxic content based on a toxicity score, which measures how offensive or disrespectful a comment is and how likely these comments could drive users away from the discussion. Aside from toxicity, there have been several studies exploring approaches towards combating fake news [\u003cspan class=\"CitationRef\"\u003e70\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e72\u003c/span\u003e] in online spaces through automated detection and moderation. Khan et al. [\u003cspan class=\"CitationRef\"\u003e73\u003c/span\u003e] explored the verification of news through a semi-automated machine learning tool to verify visual user-generated content by introducing multimedia forensics (technical analysis to detect any synthetic aspect or digital manipulation) [\u003cspan class=\"CitationRef\"\u003e74\u003c/span\u003e]. Alongside the importance of detecting such harmful content, researchers [\u003cspan class=\"CitationRef\"\u003e73\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e75\u003c/span\u003e] have stressed that it is important to understand and utilise the contextual information of such content before making any decisions.\u003c/p\u003e\n\u003cp\u003eIt is comparatively straightforward to identify extremely harmful content such as content that provides instruction for extreme weight loss or mocking victims of eating disorders [\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e76\u003c/span\u003e]. However, for some ambiguous content, the context can differentiate whether the content is harmful or harmless [\u003cspan class=\"CitationRef\"\u003e77\u003c/span\u003e]. This adds another layer of complexity for content related to body dissatisfaction and ED, as some of the ambiguous content might be contributing to negative behaviour formation in the early stages of ED, or perpetuating negative behaviours when an ED has already taken hold. Conversely, such content might be perfectly harmless for a non-vulnerable population.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGoal of this study\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe goal of this study was to explore the potential role that technology can play in reducing the negative impact of social media on body image and ED. We conducted interviews with experts by profession (professionals working in a national ED service, and body image and ED experts) and focus groups with experts with lived experience of ED. Through this paper, we provide unique insights into the perspectives of diverse stakeholders within the ED space, in the context of harmful social media content for people at-risk of or with ED. These insights are intended to inform effective content categorisation and the design of future technological solutions to mitigate the adverse effects of social media.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eWe first conducted interviews with n = 12 experts by profession. Typically a minimum sample size of 12 participants is suggested for qualitative studies in order to achieve data saturation [\u003cspan citationid=\"CR78\" class=\"CitationRef\"\u003e78\u003c/span\u003e]. Through the interviews we narrowed down the harmful content categories and their characteristics, challenges around the identification of this content and explored potential tools to mitigate the negative impact of social media. Then, with a series of 5 focus groups with n = 18 individuals with lived experience of ED (experts by lived experience), we further investigated their views on content categories relevant to ED identified by the experts by profession, and their perceptions regarding AI technologies as a potential tool for automating harmful content characterisation. AI was specifically narrowed in on for the focus groups as this was the main category of technology that the experts by profession identified in the interviews. For the experts by lived experience, we included all individuals who expressed interest in participating in the study to capture a broad range of perspectives.\u003c/p\u003e\u003cp\u003e\u003cb\u003eEthics\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe received ethical approvals from [blind for review] Human Research Ethics Committee. All participants were provided with an explanatory statement about the study and asked to sign a written consent form. Before the beginning of each session, we additionally requested that participants provide verbal consent to audio recording.\u003c/p\u003e\u003cp\u003eTo ensure the psychological safety of the lived experience participants in the focus groups, we imposed the following exclusion criteria for participation: 1) must not have an active ED, and 2) must be above 18 years old. We additionally sought in-depth feedback on all documentation and activity plans from our partner organisation (a national ED service provider in Australia), who reviewed the terminology and tone of language that we used and advised on the appropriateness of activities. They also provided us with necessary helpline services and instructions that we could provide the participants at the beginning of the focus group, to ensure adequate support provision should a participant become uncomfortable during the session. Finally, a [partner organisation] member attended the first focus group, as an observer, to ensure that our activities and the research team's facilitation were adequate. Two research team members facilitated each focus group session. Facilitators had extensive experience conducting qualitative research within the themes of mental health, ED, and digital innovation.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRecruitment\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe recruited n = 12 leading body image and ED experts across Australia, by leveraging professional networks within the research team and a snowball sampling approach (i.e. the professional networks of existing participants). We emailed advertisements to gauge interest and interested parties were provided with the explanatory statement and an opportunity to ask questions to the research team. Interviews were scheduled at a convenient time for participants, with written consent obtained prior to each session. Participants received an AUD 20 shopping voucher as thanks.\u003c/p\u003e\u003cp\u003eFor the focus groups, we recruited n = 18 individuals with lived experience of ED through [partner organisation’s] lived experience network, which encompasses people from Australia with a history of ED. Recruitment materials highlighted particular interest in speaking to traditionally underrepresented people in ED research (i.e. males, people from culturally and gender-diverse backgrounds). Interested participants filled out a Google form and were provided with the explanatory statement and an opportunity to ask questions to the research team. Focus groups were arranged at a convenient time for participants, with written consent obtained prior to each session. Participants received an AUD 60 reimbursement as thanks.\u003c/p\u003e\u003cp\u003e\u003cb\u003eInterview with experts by profession\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe conducted the interviews via Zoom which lasted approximately 1 hour. It is to be noted that when we conducted the interviews, the Australian government had not yet passed the bill for banning social media for users under 16 years old [\u003cspan citationid=\"CR79\" class=\"CitationRef\"\u003e79\u003c/span\u003e]. The participants' demographic details are provided in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The interviews focused on exploring four key areas: 1) Participants' experience and the specific populations they work with; 2) Influence of social media on body image and ED; 3) Characteristics that might be useful for identifying social media content as harmful, ambiguous and safe; and 4) Potential solutions (digital or non-digital) to mitigate the negative effects of social media on body image and ED. The interview guide can be viewed in Appendix A.\u003c/p\u003e\u003cp\u003eThrough a preliminary analysis of the interview data, supported by a review of existing literature, we generated eight content categories relevant to ED: weight loss (including content influencing changes to body shape and size), weight loss or performance-enhancing drugs, ED recovery, cosmetic surgery and other minimally invasive cosmetic procedures, beauty tutorials, food and nutrition, exercise, and body checking (i.e. where the creator is checking their body either in the camera, on weighing scales, or by measuring parts of their body).\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eDemographics of experts by profession (interviews) [Note: Participants self-identified their ethnicity or cultural background and this was not standardised by the research team]\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"6\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCode\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGender\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEthnicity or Cultural background\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eAge\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e\u003cp\u003eLived experience of ED\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c6\"\u003e\u003cp\u003eLived experience of body dissatisfaction\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt4\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e35–59\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt5\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAnglo/White\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e35–59\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt6\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eLebanese\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt7\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eBengali\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt8\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt9\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt10\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGender-diverse (Non-binary)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt11\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eYes\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInt12\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e35–59\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eNo\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003e\u003cb\u003eFocus groups with experts by lived experience of ED\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe conducted a series of 5 focus groups each lasting 1.5 hours via Zoom, with n = 18 participants. We conducted focus groups 1, 2 and 3 with females (n = 3, n = 5 and n = 6, respectively), focus group 4 with gender-diverse participants (n = 2), and finally, focus group 5 with males (n = 2). We formed these groupings to foster a comfortable discussion environment. We explicitly asked participants from focus groups 4 and 5 if they would prefer to be placed in a mixed or gender-specific grouping and they chose gender-specific. We have provided the participants' demographic details in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. The focus groups aimed to examine the role of social media in developing ED and explore potential solutions to make social media safer. We used Miro boards (an online whiteboards that allows people to collaborate in real-time using diagrams, sticky notes and other tools to write down and organise ideas) to engage with the participants. The interviews with experts by profession fed into the planning of activities for the focus group.\u003c/p\u003e\u003cp\u003eFor our first activity, we asked the participants to discuss the role that social media played at different stages of ED: early stage (i.e. where disordered eating patterns and challenging behaviours might be developing), late stage (i.e. where the ED is fully embedded and might require formal intervention), and during recovery. We gained a crucial and nuanced understanding of how certain content impacts users differently on where they are in their eating disorder journey depending on their vulnerability and context. The participants discussed both positive and negative aspects of social media for each of these stages as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u003cp\u003eFor our second activity, we asked participants to discuss as a group and create 'We need' statements for a hypothetical manifesto positioned to the government to advocate for safer social media platforms for those at-risk of or experiencing ED (inspired by methods in [\u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e]). The participants then prioritised and selected their top three statements as a group. Through this activity, we aimed to understand what kind of harm was considered as a priority by experts by lived experience, and what they considered as safety and support. By taking into account what participants say they need to feel safe and supported, we can design moderation tools that are aligned with real-world experiences.\u003c/p\u003e\u003cp\u003eDuring the interviews with experts by profession, over half of participants provided ideas for the potential solutions to mitigate the negative effects of social media on body image and ED which revolved around the use of Artificial Intelligence (AI) (e.g. use of chatbots to promote body positivity and provide automatic critical thinking tasks when certain images are detected in the social media feed). Thus, during the focus groups, we decided to explore the participants' views on AI and the potential use of AI to create safer social media. Building on this, for our third activity, we asked participants to imagine that their manifesto (created in the second activity) was accepted by the government, who had then created an AI tool embedded in social media. This activity was intended to be a provocation to yield open discussion and prompt design insights. As such, there was no description provided about how the tool functioned. This loosely followed the values of invisible design approaches, an approach which is used to generate ideas and explore insights as a form of concept development [\u003cspan citationid=\"CR81\" class=\"CitationRef\"\u003e81\u003c/span\u003e]. We asked participants to consider AI use cases, how it made them feel, how it made their life easier, potential challenges, and prominent supporters and critics.\u003c/p\u003e\u003cp\u003eFinally, participants created rules for the AI to guide content moderation by identifying and categorising harmful social media content for the eight content categories, which we derived from the interviews with the experts by profession. They were asked to list 'Must not show' (prohibited content for social media) and 'Can show' (acceptable for social media) content for each category, helping us understand their views on what content is harmful or safe for social media. The focus group guide can be viewed in Appendix B.\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eDemographics of the experts by lived experience of ED (focus group) [Note: The participants self-identified their ethnicity or cultural backgrounds and this was not standardised by the research team]\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"6\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCode\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGender\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEthnicity or Cultural background\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eAge\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c5\"\u003e\u003cp\u003eReligion\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c6\"\u003e\u003cp\u003eType of Eating Disorder (ED)\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG1P1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNA\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG1P2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian/ New Zealand nationality of Scottish descent\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eAnglican\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eBulimia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG1P3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEuropean\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNA\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG2P1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eMalaysian Chinese\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eChristian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG2P2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEuropean Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e10–24\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eAgnostic\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG2P3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAnglo-Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e10–24\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eChristian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG2P4\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEuropean\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e35–59\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eCatholic\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG2P5\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eIndian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e10–24\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eAtheist\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eCaucasian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eAtheist\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNone\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa and Binge eating disorder\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNA\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eMultiple types\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P4\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eWhite British/ Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNone\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eOther specified feeding or eating disorders (OFSED) and Orthorexia\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P5\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eCaucasian Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNA\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG3P6\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eFemale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eIraqi Muslim\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eIslam - Shia sect\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa + Avoidant/restrictive food intake disorder (AFRID)\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG4P1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGender-diverse (Transgender man)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAustralian with a Greek cultural background\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNone\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG4P2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eGender-diverse (Non-binary)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eMaori, European\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNone\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eBulimia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG5P1\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSri Lankan, Australian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e25–34\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eBuddhist\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFG5P2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMale\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eCaucasian\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e35–59\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003eNone\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u003cp\u003eAnorexia nervosa\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003ch2\u003eData Analysis\u003c/h2\u003e\u003cp\u003eAcross the study, we had 19.5 hours of audio-recorded data. The first author transcribed all the audio recordings and de-identified the transcripts. After the completion of the interviews with experts by profession, the first author used an inductive thematic analysis method [\u003cspan citationid=\"CR82\" class=\"CitationRef\"\u003e82\u003c/span\u003e] to generate themes and codes at the paragraph-to-sentence level without any pre-existing theoretical framework. NVIVO was used to code the transcripts. The first author generated the initial codes and the remaining research team familiarised themselves with the data from the experts by profession and reviewed the initial codes. After iterative coding, scoping of the existing literature and numerous discussions a set of content categories was generated. These categories were further explored in the focus groups with experts by lived experience. After the completion of the focus groups, the first author again used inductive thematic analysis [\u003cspan citationid=\"CR82\" class=\"CitationRef\"\u003e82\u003c/span\u003e] to generate initial codes and themes which were reviewed and discussed in an iterative process by the research team. Any discrepancies were resolved through discussion until a consensus was reached on the final significant themes.\u003c/p\u003e\u003cp\u003eData from the existing literature, interviews and focus groups were triangulated and are presented as such in the findings [\u003cspan citationid=\"CR83\" class=\"CitationRef\"\u003e83\u003c/span\u003e]. The interviews provided in-depth academic and professional knowledge and reasoning behind harmful content categorisation, as well as insights around opportunities for technology to mitigate the negative effects of social media. The focus groups then added a crucial layer of nuance by providing personal perspectives from those with lived experience of ED. The triangulation confirmed emerging themes and increased the credibility of the findings [\u003cspan citationid=\"CR84\" class=\"CitationRef\"\u003e84\u003c/span\u003e].\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eIn this section, we will consolidate the findings from interviews and focus groups. For easy identification, we have added prefixes to the participants of interviews and focus groups: interviews (Int) and focus group (FG). Throughout our findings, we refer to participants from interviews as experts by profession and focus groups as experts by lived experience, to further add clarity regarding where data was derived. Six core themes from across the data are reported below: 1) Understanding contextual factors of social media content, 2) Contributing factors to the ED echo chamber, 3) Challenges for content moderation in social media, 4) Needs and requirements of stakeholders for a safer social media experience, 5) Promoting diversity in social media, and 6) Perceptions regarding use of technology to mitigate the negative impact of social media.\u003c/p\u003e\u003cp\u003e\u003cb\u003eUnderstanding contextual factors of social media content\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe experts by profession stressed that understanding social media content can be nuanced and subjective, which becomes a challenge in differentiating between harmful and safe content; \u003cem\u003e\"the fact that it's not black and white. [...] People might have different perspectives on whether something says red [harmful] or amber [ambiguous] or even green [safe]\"\u003c/em\u003e (Int12) and \u003cem\u003e\"I guess this is the problem because it wouldn't be harmful for everybody [...] So, there would be a lot of people that look at that content and would be absolutely fine and not harmed at all.\"\u003c/em\u003e (Int2). Int2 suggested that to overcome this challenge, the potential solution could be to separate harmful content categorisation for different population groups: \u003cem\u003e\"whether we categorise as harmful for everyone or harmful for people 'at-risk'. So, for example, people with ED, people at-risk of ED, or people in recovery could be part of that kind of 'at-risk' group, and then you have kind of a general one\"\u003c/em\u003e (Int2). Despite nuances and subjectivity, there was clear value in consolidating the dispersed ideas of what constitutes 'harmful' content and developing a standardised, consensus-based list \u003cem\u003e\"Thank you for doing this work. I do think it's important to pull it all together [..] if we can have a consensus on this list that's going to be really powerful\"\u003c/em\u003e (Int5).\u003c/p\u003e\u003cp\u003eParticipants broadly discussed a range of different social media content types, from generalised content that might be considered normal viewing on social media platforms (e.g. healthy eating recipes that provide nutritional guidance, exercise videos, and beauty tutorials) to more obviously challenging content (e.g. the promotion of weight loss drugs and cosmetic procedures, very underweight creators performing body checking behaviours - weighing, measuring, pinching parts of their body, or generally looking for signs of visible fat). Concerning more generalised content, which could be considered ambiguous in its impact, the experts by profession agreed that the intention of the video was vital:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"They [fitness influencers] are more ambiguous because while people may take that as inspiration and use that for a healthier lifestyle to get more in shape and to have just a better physical health, it can be negative for those who are obsessed with exercise and engage in excessive exercise that is not necessary for them. And a lot of that exercise is driven by appearance more so than driven by health\"\u003c/em\u003e (Int7).\u003c/p\u003e\u003cp\u003eThe experts by profession suggested that ambiguous content starts moving towards harmful when it starts focusing towards comparisons like 'before and after': \u003cem\u003e\"we find that [before and after] really toxic obviously because they're saying that in the first image they're not worthy. They're not worthy unless they lose weight and head towards that after image\"\u003c/em\u003e (Int11) and content promoting very low-calorie diets: \u003cem\u003e\"Some content might have meal plans that follow really low-calorie meal plan [...] if people are encouraged to follow that plan to lose weight or because they're not worthy of their current appearance, that can be really damaging\"\u003c/em\u003e (Int11). In addition, accompanying textual representations of elements such as the creator's weight or body measurements, the calorie content of foods, and calories burned, were often seen to be the potentially harmful element, even if the overall tone of the video was more positive: \u003cem\u003e\"Anything that shows numbers like if someone talk about how many kilograms they weigh and what their goal weights are, or showing how many calories they might be eating, or how many grams of sugar\"\u003c/em\u003e (Int5) and \u003cem\u003e\"I think stuff using quantitative numbers, things like that should probably be banned. Anything where someone says their weight or calorie limits should be banned\"\u003c/em\u003e (FG2P3). The experts by lived experience further clarified why showing numbers in such content can be so damaging: \u003cem\u003e\"As somebody who weighed their food and counting calories for years, it took forever to get out of that[...] a lot of the times these people are guesstimating calories. It just doesn't need to be there\"\u003c/em\u003e (FG2P5). Perhaps the most extreme form of content was that depicting body-checking behaviours from creators who would be considered underweight: \u003cem\u003e\"encouraging people to take their headphone cord and see if they could wrap it around to their stomach and if they were able to reach the full head headphone cord that was good, they were skinny\"\u003c/em\u003e (Int11).\u003c/p\u003e\u003cp\u003e'What I eat in a day' videos (a common trend on social media depicting videos around what a person eats throughout the day) were seen as particularly challenging across the entire cohort of participants: \u003cem\u003e\"because everyone's different. What might be sufficient for me to eat in one day may not be sufficient for someone else to eat in one day. It doesn't take anyone's history or medical conditions or cultural aspects in consideration\"\u003c/em\u003e (FG3P6). Participants stressed that any type of diet plan or health advice should come from experts with proper credentials, or have a disclaimer that it is not applicable for all:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"even if someone is qualified to be giving that advice, they don't know who's going to be taking it in\"\u003c/em\u003e (Int11); and \u003cem\u003e\"a lot of these health gurus that might not even be trained in certain areas about food or nutrition and like targeting a lot of vulnerable people to buy certain products which can also lead to body image concerns\"\u003c/em\u003e (FG4P1).\u003c/p\u003e\u003cp\u003eThe experts by profession additionally highlighted several other significant body-related trends, such as the 'A4 Waist Challenge': \u003cem\u003e\"A trend that involved getting an A4 piece of paper and holding that up in front of your body as well. And if your body was larger than that vertical, a four piece of paper, you would be shamed. If it was smaller, you would obviously be praised and considered that's an ideal\"\u003c/em\u003e (Int11) and the promotion of thigh gaps as a body ideal: \u003cem\u003e\"Thigh gaps, which I think now has kind of transitioned into legging legs. [...] do you have like a big thigh gap, and they're just like straight twigs?\"\u003c/em\u003e (Int1).\u003c/p\u003e\u003cp\u003eContent promoting weight loss drugs (like Ozempic and Mounjaro), or performance-enhancing steroids, to aid in increasing muscle mass, were seen to be extremely harmful and often facilitated the spread of misleading information, that could put one's health at risk:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"If someone, for example, is saying \"Oh!! take Turkesterone [help increase muscle mass and strength], it'll boost your testosterone, you'll get mad gains\", there's zero evidence for the safety and efficacy of that. Therefore, that is misinformation, and that kind of content is harmful either on misleading ground or because it's intensely negative\"\u003c/em\u003e (Int9).\u003c/p\u003e\u003cp\u003eThe experts by lived experience were vehemently against showcasing any advertisements related to weight loss drugs: \u003cem\u003e\"It will put them [any viewers] at risk [...] I don't know why an advertisement would be needed\"\u003c/em\u003e (FG3P2) and \u003cem\u003e\"I would love to see that just not exist on social media personally\"\u003c/em\u003e (FG4P2).\u003c/p\u003e\u003cp\u003eWhen it came to content discussing ED recovery, there were mixed views. Participants made it clear that ED recovery content showcasing medical components or hospital settings should be considered harmful, as it provides a source for comparison of symptoms. If not presented carefully, this could be considered triggering, or indeed a motivator for people at the earlier stages of their ED: \u003cem\u003e\"Any content that is based in a hospital setting or implies that someone has to go to a hospital for their ED to be taken seriously, that would obviously be toxic\"\u003c/em\u003e (Int11) and \u003cem\u003e\"They're doing XYZ, so this means that they have an ED, whereas I don't have XYZ symptoms or I don't have those food rules. So, therefore maybe I don't have an ED. I'm just spiralling for no reason and I don't actually have to get help\u003c/em\u003e\" (FG2P5).\u003c/p\u003e\u003cp\u003eFinally, a surprising source of ambiguous content that was discussed was beauty tutorials. While beauty tutorials don't directly impact ED, they can cause body dissatisfaction and strengthen the 'idealised' concept that can cause serious harm to society:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"We've seen an influx of 12-years-old going into Mecca and Sephora and wanting to buy retinol [anti-ageing product] and ingredients that are just way beyond their age. [...] We often think about body image as being something from the neck down. But, I think social media has tried to include everything\"\u003c/em\u003e (Int4) and \u003cem\u003e\"Definitely harmful, obviously for emphasising that appearance is super important\u003c/em\u003e\" (FG3P3).\u003c/p\u003e\u003cp\u003eThe drive to change the way one looks, to attain an 'ideal', was also seen to potentially lead to more drastic steps, such as unnecessary cosmetic procedures: \u003cem\u003e\"content that encourages dieting or, things like unnecessary cosmetic or medical procedures to appear a particular way\"\u003c/em\u003e (Int4). The experts by lived experience stressed the exploitative nature of this type of content: \u003cem\u003e\"it's driven by exploiting someone's emotional state. It's all hype and bubble [...] it shouldn't be in the hands of someone who's making profit\"\u003c/em\u003e (FG5P2).\u003c/p\u003e\u003cp\u003e\u003cb\u003eContributing factors to the ED \"echo chamber\"\u003c/b\u003e\u003c/p\u003e\u003cp\u003eAn ED \"echo chamber\" refers to a digital space where content normalising and promoting disordered eating behaviours are continuously shared, such that the individuals in the space are repeatedly exposed to content which further reinforces their disordered eating beliefs and behaviours. Participants in all phases discussed the development of an echo chamber of sorts, created through continuous viewing of certain content types, which served to normalise unrealistic body standards and feed into existing belief systems: \u003cem\u003e\"it's going to create these appearance norms where people think 'Ohh!! Everyone looks like this and I should look like this'. So it's adding to these pressures, both in terms of the body size and facial appearance\"\u003c/em\u003e (Int1) and \u003cem\u003e\"The more we're exposed to them [unrealistic ideals], I think the more they're perceived as normal\"\u003c/em\u003e (FG1P1). The experts by lived experience brought a new perspective around how certain types of content could create a sense of competition in individuals at-risk of or with ED: \u003cem\u003e\"There is that culture on social media of competition to see who can be the skinniest, who can recover the fastest, or who can do whatever. It's a real competition to keep up with those things\"\u003c/em\u003e (FG2P4).\u003c/p\u003e\u003cp\u003eThe echo chamber of ED content starts to blur the boundary between reality and harmful ideals, with filters and photo-shopped content playing a leading role in perpetuating unrealistic standards: \u003cem\u003e\"As a society as a whole, the less realistic, the less attainable, the more photo-shopped, the more filtered tends to be unhelpful because it's portraying something that's not actually real or achievable for the vast majority of people\"\u003c/em\u003e (Int12). It distorts reality to the point that, when individuals come face-to-face with reality, they cannot accept it:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I got at a certain point, I kind of got used to the filtered version of me. So when I looked in the mirror and saw the real version, it was quite upsetting for me and also that sense of discomfort of not being able to post an unfiltered picture when you get used to the filtered version of yourself\"\u003c/em\u003e (FG4P2).\u003c/p\u003e\u003cp\u003eThe disconnect between filtered social media content and how individuals appear can lead to people feeling as though they don't measure up: \u003cem\u003e\"with filters to hide cellulite or wrinkles or whatever it might be. I think for those vulnerable people continually seeing these types of images makes them feel as though they're sort of outside of the box if they don't fit that mould\"\u003c/em\u003e (FG1P3).\u003c/p\u003e\u003cp\u003eThe inherent harm of social media algorithms within this context further highlighted this point. If viewers are watching certain types of harmful content, algorithms are learning this pattern and consequently exposing them to similar content, creating a feedback loop that intensifies harmful narratives: \u003cem\u003e\"It's not just the content, it's the algorithms that recommend and deliver content and the ways that people engage with that content, whether consciously or unconsciously, that makes it more likely that the content will be delivered to them\"\u003c/em\u003e (Int9). Existing biases in the algorithm itself can also serve to further strengthen the likelihood of harmful content reaching certain demographics:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"the algorithm is gender biased, so it will show young girls who have not searched for anything or followed anything in relation to diet and nutrition, but it will show girls diet content very early on in their journey on the platform\u003c/em\u003e\" (Int5) and \u003cem\u003e\"there have been some suggestions that there is some dark stuff happening around censoring non-white creatives\"\u003c/em\u003e (Int1).\u003c/p\u003e\u003cp\u003eTrying to combat this echo chamber, created by the algorithm, was something the experts by lived experience found significantly challenging, particularly in the recovery process. The continuous bombardment of such content threatened, and potentially penetrated, the shield they had formed against these concepts:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"There is this whole thing of constant everyday battle with your algorithm. As soon as you open anything [...] and no matter what you're doing, you're still opening up feeds that are going to have things in it [harmful], so you're still put in that space where you're having to combat and filter and challenge all of these messages. So, there's still a big drain that comes from it\"\u003c/em\u003e (FG1P1).\u003c/p\u003e\u003cp\u003eFor some, this led to a withdrawal from social media altogether in an attempt to protect their recovery.\u003c/p\u003e\u003cp\u003eAs discussed by the experts by profession, comments left by other social media viewers on creators' content were often seen to be as harmful as the content itself: \u003cem\u003e\"I find with a lot of toxic social media content, especially on TikTok, you'll often get comments as well that are just as toxic, if not more toxic than the actual video.\"\u003c/em\u003e (Int11). These comments can amplify negative views surrounding body image, acting somewhat as evidence that a person is not good enough in the eyes of society: \u003cem\u003e\"the comments that come through is a very clear indicator of the public perception or opinion on whatever this main topic is\"\u003c/em\u003e (Int6) and \u003cem\u003e\"you're seeing someone else getting negative comments who doesn't make those ideals and that can make you feel negative, even if those comments are not directed at you specifically\"\u003c/em\u003e (Int10). Even when the content itself was seen to have positive components, such as promoting encouragement to live a healthier lifestyle, the negative comments were seen to overpower the positivity of the content itself. This phenomenon was particularly notable in content made by creators living in larger bodies: \u003cem\u003e\"The post that generates those kinds of conversation can be a positive post like it could be a fat person enjoying life or a fat person getting married or succeeding, but there will still be comments underneath that are fat phobic, negative and abusive towards that person\"\u003c/em\u003e (FG4P2).\u003c/p\u003e\u003cp\u003e\u003cb\u003eChallenges for content moderation in social media\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe experts by profession expressed frustration over their experiences with content reporting, with the manual scouring and flagging of content a resource-intensive process:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"And so they [social media platforms] come and say we need more information, that is really time and resource heavy. It takes a long time to then go back through especially if you need to go back through lots of different videos and comments. [...] Certainly, the time and manual aspect of it would be a challenge\"\u003c/em\u003e (Int11).\u003c/p\u003e\u003cp\u003eIt was often the case that social media platforms refused to remove harmful content, citing their platform guidelines as a justification rather than trusting the expert opinions of ED specialists:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"Someone in our community found this person on TikTok who was doing these years of body checking, and this person was really, really underweight [...] quite a lot of the comments were full with people saying 'give me your diet tips' - that sort of thing. And her account was just dedicated towards these body checks. However, she didn't mention eating disorders in the caption and she didn't talk about eating disorders at all. It was only the people commenting that mentioned eating disorders. Since she didn't mention eating disorders, TikTok said that it did not go against their guidelines\"\u003c/em\u003e (Int11).\u003c/p\u003e\u003cp\u003eOf course, in many cases, particularly with more ambiguous content, creators are not aware that their content could be harmful to people experiencing or at-risk of ED. However, in others, creators deliberately try to evade social media safeguards to keep their harmful content online, through methods such as mutation or avoidance of hashtags: \u003cem\u003e\"People mutate those hashtags. So '#anorexia' might now be displayed as [an0r3xia], but the 'O' is a zero or the the 'E' is three\"\u003c/em\u003e (Int11) and \u003cem\u003e\"A lot of people on Twitter will post what is ostensible 'thinspiration' and not hashtag it with particular things because they know that if people see it, who might not have an active eating disorder or be actively kind of restricting, they'll report it and they'll get removed.\"\u003c/em\u003e (Int8). Even banning certain harmful hashtags does not guarantee proper blockage of harmful content. The experts by lived experience drew this concept out further, sharing incidents of 'hashtag hijacking':\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I think it can be difficult when hashtags that are meant to be helpful, especially for hashtags saying recovery for eating disorders are hijacked by influencers or people promoting things like weight loss. [...] Concepts of 'intuitive eating' is like 'intuitive fasting' now and that's really unhelpful\"\u003c/em\u003e (FG3P4).\u003c/p\u003e\u003cp\u003eThere is a distinct tension surrounding social media platforms in regards to maintaining a safe online environment for viewers and not implicating freedom of speech:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I think the freedom of speech is definitely a big one and definitely something that I've felt from the respective apps [social media platforms]. That is a very big concern of theirs and they want to keep that in check so they're not censoring their users to such an extent where they will go to a different app to share their story and talk about the same things\"\u003c/em\u003e (Int11).\u003c/p\u003e\u003cp\u003eThis freedom of speech differs from country to country, which might pose a challenge for content moderation more broadly:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"There's a cultural element to that as well, and that a lot of these companies are American-based where the cultural component of freedom of speech is so strong and powerful. I think in Australia, we're more used to being censored, and we're more comfortable with the benefit that can have for protecting vulnerable people. But I think in America the swing is towards that freedom of speech element\"\u003c/em\u003e (Int12).\u003c/p\u003e\u003cp\u003eHowever, the experts by lived experience were adamant that 'freedom of speech' should not be an excuse provided by content creators and platforms to push harmful content, that could ultimately affect someone's life:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"All the pushbacks towards governments trying to introduce greater punishments and regulation for these platforms in the name of freedom of speech, but the big argument is if it impacts people's mental health badly then you can't cause millions of people [harm] by affecting their mental health and get away with it completely. There needs to be some sort of accountability for both the creators and platforms\"\u003c/em\u003e (FG2P3).\u003c/p\u003e\u003cp\u003eParticipants felt that the corporations and content creators who are beneficiaries of diet culture and societal beauty standards, and who rely on advertisements and content reach to promote products could potentially be the barrier for resistance to content moderation:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I think a lot of people would have difficulty with being told that they can't post certain things or seek out certain information. [..] I think the diet culture is so prevalent that fills the pockets of industries. Individuals will struggle to adhere to the rules\"\u003c/em\u003e (FG3P1) and \u003cem\u003e\"People, organisations and companies whose income and profit is based on promoting harmful content on social media [...] I think would be quite a critic here\"\u003c/em\u003e (FG5P1).\u003c/p\u003e\u003cp\u003e\u003cb\u003eNeeds and requirements of stakeholders for a safer social media experience\u003c/b\u003e\u003c/p\u003e\u003cp\u003eParticipants in all phases mentioned the need to bring government and social media platforms into the agenda to make social media space safer. The needs ranged from making the government aware of the situation \u003cem\u003e\"can we go to policymakers and say that this type of content is either safe or ambiguous or might help to mitigate some of the onslaught of idealised content that's pushed to these demographics\"\u003c/em\u003e (Int4) and making it compulsory for platforms to open their API (Application programming interface) for research \u003cem\u003e\"By giving us access to things like the API and, within reason, how the algorithms work, research can be more specific... higher quality; and policies that are implementable and useful and that don't put undue burden on platforms can be brought into being\"\u003c/em\u003e (Int9). Additionally, Int9 also discussed how researchers could aid the government in bringing platforms into the loop, by providing them with solid evidence:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"The more specific we make our research and the better we can articulate mechanisms, the easier it is for policymakers to move forward, because if we present them with findings that are tantamount to, 'social media, on average is weakly positively correlated with body dissatisfaction' - What can they do with it? They can't ban social media. They don't know what to even tell the social media companies. So that's the best thing we can do is to push ourselves for methodological complexity and specificity in the research\"\u003c/em\u003e (Int9).\u003c/p\u003e\u003cp\u003eAll participants agreed that there was value to social media, and future interventions needed to work with, rather than against, the benefits it could provide: \u003cem\u003e\"They're [young people] born in the age of social media. We just need to work out\"\u003c/em\u003e (FG2P4) and \u003cem\u003e\"because then you miss out on the positives that social media provides as well. [...] you lack that connection with certain people and stuff like that\"\u003c/em\u003e (FG3P1). The experts by profession, in particular, highlighted how responsible content creators were useful vehicles for promoting positive behaviours: \u003cem\u003e\"They [responsible fitness content creators] do have ideal bodies themselves [...] but they are doing it in the most healthful way that can be done. And what I like about that content is that it's practical and has a really significant reach\"\u003c/em\u003e (Int9).\u003c/p\u003e\u003cp\u003eThe experts by profession also emphasised the necessity of collaborating with content creators to promote responsible content generation: \u003cem\u003e\"I think that would be really great if we could see a culture shift amongst influencers and content creators as to what they're posting, how they're posting it and if it's done in a safe way\"\u003c/em\u003e (Int11). However, the experts by lived experience described that current safety measures that content creators often provide were not enough: \u003cem\u003e\"I agree with filtering out things before they get to being in front of the person cause things like trigger warnings and blurred posts don't do anything\"\u003c/em\u003e (FG3P1).\u003c/p\u003e\u003cp\u003e\u003cb\u003ePromoting diversity on social media\u003c/b\u003e\u003c/p\u003e\u003cp\u003eParticipants in all the phases stressed that it was necessary to see more diversity in social media content, in terms of body shape, size, colour and gender: \u003cem\u003e\"I think it's healthy to see diversity of body shape and size\"\u003c/em\u003e (Int12). Diversity is essential to normalise realistic body standards and create a sense of belonging and connection for the viewers. This sense of belonging is particularly significant for ED recovery:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I didn't see a single person of colour talking about their ED. And I think that just definitely felt like I was alienated even more\"\u003c/em\u003e (FG2P5) and \u003cem\u003e\"I wasn't able to see myself represented anywhere in stories of hope. So, then I didn't think there was hope when I was in recovery\"\u003c/em\u003e (FG4P1).\u003c/p\u003e\u003cp\u003eThis concept of diversity needs to be more broadly expanded towards underrepresented populations in the ED space (culturally diverse, males, gender-diverse and differently-abled people):\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"it was females who were like White and upper class and that they were the people that I'd see share their stories, but I never really saw any diversity of people with disabilities or men or people with different gender and sexuality or different race or homeless or anything like that\"\u003c/em\u003e (FG4P1).\u003c/p\u003e\u003cp\u003eThe experts by lived experience also shared how not being represented in promotional materials for support services on social media could create a barrier for seeking help: \u003cem\u003e\"A lot of body neutrality posts are geared towards young women. I think a lot of support services are geared towards young women as well [...] I held a lot of shame around accessing services because I was a guy who was experiencing something like that\"\u003c/em\u003e (FG5P1).\u003c/p\u003e\u003cp\u003eThe lack of representation becomes further complicated when intersectionality comes into play:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"When you're a person who experiences multiple types of disadvantages. So, for me, it was my body size, you know, being a fat person, but other things like my ethnicity, my facial appearance my body shape, socioeconomic and all other things, I think it almost placed a greater pressure on me to try and be beautiful because I can't change these others things about myself [...] it felt like the only the only way I would ever be able to join the realm of the acceptableness\"\u003c/em\u003e (FG4P2).\u003c/p\u003e\u003cp\u003e\u003cb\u003ePerceptions regarding use of technology to mitigate the negative impact of social media\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWhen asked about the technologies that might aid in mitigating negative impacts of social media, participants discussed the value of automation to support the time-intensive process of moderation: \u003cem\u003e\"AI could go in and kind of just flag anything that could potentially harmful, so that as a moderator, you're not having to constantly go through because it's time consuming\"\u003c/em\u003e (Int4) and \u003cem\u003e\"If it [AI] could identify that kind of content [harmful] and remove it completely\"\u003c/em\u003e (FG2P1). The experts by lived experience additionally suggested an option for filter removal: \u003cem\u003e\"remove like filters that alter your body and appearance so then everyone comes through as who they actually are\"\u003c/em\u003e (FG4P1).\u003c/p\u003e\u003cp\u003eThe experts by profession also valued the potential screening and tracking capabilities of technology, for automating the detection of viewers' behaviours: \u003cem\u003e\"detecting that there's an issue for someone by looking at their patterns of usage of online social media\"\u003c/em\u003e (Int6) and identifying patterns of viewing harmful content: \u003cem\u003e\"If you can see this, you can present this to the user and you can try to deliver notifications that move people somewhere else\"\u003c/em\u003e (Int9). The experts by lived experience had similar thoughts: \u003cem\u003e\"screen if someone is searching for particular content [harmful] and maybe alert a psychologist that they might need support or follow-up\"\u003c/em\u003e (FG2P2).\u003c/p\u003e\u003cp\u003eFor the experts by lived experience, having a level of personalisation was vital to support individual needs: \u003cem\u003e\"like having some personalisation so that it can be customisable because it's very different for everyone\"\u003c/em\u003e (FG4P2), suggesting a form of of feedback system to support better recommender systems: \u003cem\u003e\"Maybe if afterwards or during [watching content], you've got the option of AI asking, 'Is there something you didn't like in this video? That you want us to hide in the future'.\"\u003c/em\u003e (FG1P3). In their view, using technology with added safeguards would give them back control and make them feel safer:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I think it would make me feel more willing to spend time online and engaging in online spaces. There's so much stuff that I don't have control over, and it's not like I don't know.\"\u003c/em\u003e (FG1P2); \u003cem\u003e\"Probably feel more at peace, like less consumed by like all this misinformation and be able to trust myself more\"\u003c/em\u003e (FG4P1); and \u003cem\u003e\"Well, mine would probably be AI that alters the algorithm to show people content that uplifts them\"\u003c/em\u003e (FG4P2).\u003c/p\u003e\u003cp\u003eParticipants also saw this as a chance to use algorithmic manipulation to populate social media feeds with positive and happy content: \u003cem\u003e\"you limit to only the stuff that brings you joy, the stuff that captures your attention or keeps you watching that brings you joy and we're just going to show you that\"\u003c/em\u003e (Int1); and \u003cem\u003e\"Anyone who\u0026rsquo;s coming up with this thing [AI solution], would have spent enough time to work out the plotted chart of where it leads to. So, if that information is available to people to know where it\u0026rsquo;s going and what is the intention behind it\"\u003c/em\u003e (FG5P2).\u003c/p\u003e\u003cp\u003eHowever, the experts by profession had some concerns regarding the technology itself in terms of its performance, due to the subjectivity of what might be considered harmful: \u003cem\u003e\"I worry that the machine learning wouldn't be able to distinguish between stuff that was harmful and not harmful because there's such a nuance to that. I think probably as I've been talking and it's very hard to quantify what is actually harmful or not\"\u003c/em\u003e (Int10). They emphasised the need for human intervention in specific scenarios, rather than relying solely on the technology to take specific actions: \u003cem\u003e\"I guess ideally if someone is expressing very troubling things to a chatbot [technology], it should sort of connect them with someone that they can speak to, like a real person\"\u003c/em\u003e (Int3). Int3 further added that technology has to be appropriately evaluated and validated before use: \u003cem\u003e\"As long as there was lots of piloting of it, then I think I would feel OK\"\u003c/em\u003e (Int10). There were also some concerns from the experts by lived experience regarding using AI in clinical settings:\u003c/p\u003e\u003cp\u003e\u003cem\u003e\"I have concerns around how AI is used in diagnosis and treatment. But, I think I would perhaps look at AI being like the prevention side of things and identify the systems that currently exist that are creating harmful environments for people around their body image and ED, or maybe at the risk of ED\"\u003c/em\u003e (FG5P1).\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eOur study aimed to explore the different contextual factors within harmful social media content for people at-risk of or experiencing ED, inform content moderation strategies and assess the potential role of technology in reducing the negative impact of social media. Our discussion is structured as follows. First, we discuss how there is a need for shared responsibility between different key stakeholders and how technology can support that shared responsibility in content moderation. We then describe opportunities for integrating diversity and culture into social media, to create a more inclusive and diverse environment. Finally, we explore how the use of understanding these nuanced contexts could support novel approaches towards the design of future interventions to protect social media users experiencing, or at-risk, of eating disorders from harm.\u003c/p\u003e\u003cp\u003e\u003cb\u003ePrincipal Findings\u003c/b\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eNeed for shared responsibility in content moderation\u003c/b\u003e\u003c/p\u003e\u003cp\u003eOur findings indicate that filtering harmful content should not fall solely on individuals with ED or their supporters. Shared responsibility from users, creators, platforms, and government is essential. There is potential in leveraging the inherent design of social media algorithms, which optimises user engagement by curating tailored content. Algorithmic manipulation might involve viewers proactively taking steps to curate their social media feeds through 'digital pruning' [\u003cspan citationid=\"CR85\" class=\"CitationRef\"\u003e85\u003c/span\u003e]. Unfollowing, muting or blocking harmful accounts and content, and critically thinking about harmful content and its potential impact is an essential aspect of media literacy [\u003cspan citationid=\"CR86\" class=\"CitationRef\"\u003e86\u003c/span\u003e]. By deliberately following content or creators that don't focus on appearance (e.g. instead focusing on content that aligns with one's hobbies and interests) [\u003cspan citationid=\"CR87\" class=\"CitationRef\"\u003e87\u003c/span\u003e], users can reduce the recommendation of 'idealised' content and provide a greater sense of control and agency. There is a significant body of work which has developed media literacy programs [\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e, \u003cspan citationid=\"CR88\" class=\"CitationRef\"\u003e88\u003c/span\u003e], however as our participants stressed, focusing on only the users to curate their content and think about harmful content in a critical way is not enough, as this runs the risk of exposing potentially vulnerable users to harmful content in the first place, which is a known risk factor for ED formation and exacerbation [\u003cspan citationid=\"CR89\" class=\"CitationRef\"\u003e89\u003c/span\u003e, \u003cspan citationid=\"CR90\" class=\"CitationRef\"\u003e90\u003c/span\u003e]. To ease the burden on the individual, we could potentially use automated technology such as mobile sensing approaches and automated monitoring of social media behaviours (e.g.Facebook's suicide prevention algorithm which browses through user posts to detect potential suicide risks [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]). The experts by lived experience in our study mentioned that they were not apprehensive to use automated social media monitoring if it meant that they would be safe from harmful content. Similarly, Vega et al. [\u003cspan citationid=\"CR91\" class=\"CitationRef\"\u003e91\u003c/span\u003e] reported that their participants with binge eating and bulimia showed high levels of acceptance for automated sensing approaches and engagement with reflective activities and contextual logging. The findings of this study align with our results, highlighting the potential of context-aware systems in monitoring and reflecting on social media behaviours which could be a promising area for further research.\u003c/p\u003e\u003cp\u003eThe Australian government is acutely aware of the dangers of social media for children and young people, which can be seen through the ban of social media for children below 16 years [\u003cspan citationid=\"CR79\" class=\"CitationRef\"\u003e79\u003c/span\u003e]. While this ban emphasises a responsibility for social media platforms to actively engage in safeguarding vulnerable populations from online harms, it also underscored the broader need for shared responsibility. Aside from age-based protections, it is crucial to address the needs of other vulnerable groups such as individuals who are at-risk of or experiencing ED and who may fall outside the \u0026lsquo;under 16\u0026rsquo; age bracket. Ensuring online safety for all users requires a collaborative approach, where both government policy and platform accountability come together to address the online harm.\u003c/p\u003e\u003cp\u003eAs our findings showed, the reality is that despite features like unfollowing, muting and blocking in platforms, loopholes in platform guidelines still allow some creators to bypass moderation systems. These bypasses can lead to viewers being recommended similar content again [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. As a solution, platforms could provide a 'reset' algorithm option, which would help the algorithm unlearn patterns and behaviours of the viewers and start afresh. For example, if an individual is diagnosed with ED or finds themselves in a period of ED relapse, then they can reset their social media algorithm rather than cutting off social media altogether. As expressed by our lived experience participants, this could be especially valuable during the recovery stage of ED, when users are particularly vulnerable to the impacts of harmful content.\u003c/p\u003e\u003cp\u003eOne factor that future researchers conducting content moderation work need to be mindful of is that moderation efforts often face criticism under the banner of 'freedom of speech'. There is an ongoing battle between 'freedom of speech' and public safety and well-being. Kozyreva et al. [\u003cspan citationid=\"CR92\" class=\"CitationRef\"\u003e92\u003c/span\u003e] explored the critical factors that can tip the scales between these conflicting interests: the extent of harm, frequency and repetition of conducting harm, and the content category. Creating blanket bans on content categories is highly infeasible as without proper critical thinking skills regarding its harm, users will figure out ways to circumvent blanket bans [\u003cspan citationid=\"CR66\" class=\"CitationRef\"\u003e66\u003c/span\u003e]. Such blanket bans could potentially censor recovery content that could be highly motivating and useful for individuals at-risk of or experiencing ED [\u003cspan citationid=\"CR93\" class=\"CitationRef\"\u003e93\u003c/span\u003e]. Instead, future research could focus on supporting users and moderators (like our experts by profession) to capture relevant data and automate some of the time-consuming processes such as personalised report generation for findings, focus on automated data collection rather than self-reporting, so that they can present solid cases to social media platforms for required content bans. This type of supported data collection may also prove useful for generating reports that drive government support for restrictions on certain social media content. For example, in a report from 'Reset Australia' [\u003cspan citationid=\"CR94\" class=\"CitationRef\"\u003e94\u003c/span\u003e], the authors presented an experimental study where paid-for-advertisement approval systems from various platforms approved their pro-ED advertisement. Such easy bypasses expose vulnerable viewers to harmful content, and studies like this show how easy it is to slip through existing content moderation processes. Building technologies that support evidence collection (e.g. tracking and analysing approval processes for ads, interrogating and analysing platform guidelines and extracting comments from social media posts) could be a significantly valuable contribution, providing evidence that would support governments to take action against social media platforms that allow such content.\u003c/p\u003e\u003cp\u003e\u003cb\u003eEnhancing diversity to break out of the echo chamber\u003c/b\u003e\u003c/p\u003e\u003cp\u003eOur findings showcased the importance of bringing diversity (in terms of body shape and size, culture, gender, and ability status) into social media feeds to help support an escape from the echo chambers that social media algorithms create. Enhancing diversity not only serves to blur the lines between what is considered 'ideal' and the reality of how society is made up, but also helps people who might be outside of the stereotypical white, female and anorexic vision of an ED [\u003cspan citationid=\"CR95\" class=\"CitationRef\"\u003e95\u003c/span\u003e] feel connected during ED recovery. However, like many AI-driven algorithms, social media algorithms are inherently biased. The findings from our research resonate with multiple research studies, which have found social media algorithms to be biased towards creators with disabilities [\u003cspan citationid=\"CR96\" class=\"CitationRef\"\u003e96\u003c/span\u003e], larger bodies [\u003cspan citationid=\"CR97\" class=\"CitationRef\"\u003e97\u003c/span\u003e], from different racial backgrounds [\u003cspan citationid=\"CR98\" class=\"CitationRef\"\u003e98\u003c/span\u003e] and from LGBTQ\u0026thinsp;+\u0026thinsp;communities [\u003cspan citationid=\"CR99\" class=\"CitationRef\"\u003e99\u003c/span\u003e]. This bias, which sidelines creators from diverse backgrounds, can limit viewers' exposure to content that might reflect their own identities (e.g. gender, sexuality, physical appearance, body functionality and cultural background). Thus there is a need to provide access to marginalised content creators in order to normalise variance and avoid confinement towards restrictive beauty standards. Recent work by Shrestha et al. [\u003cspan citationid=\"CR100\" class=\"CitationRef\"\u003e100\u003c/span\u003e], which co-designed digital interventions for body image with underrepresented populations, highlighted the vital importance that exposing individuals to diverse bodies can have in promoting positive body image. Their participants showed high enthusiasm for future interventions that supported increased diversity in social media (in terms of representation of diverse creators and non-appearance-related content).\u003c/p\u003e\u003cp\u003eBiases can also be addressed through using diverse datasets during model training to minimise gender and cultural biases [\u003cspan citationid=\"CR101\" class=\"CitationRef\"\u003e101\u003c/span\u003e]. AI developers, in particular, should provide specific considerations to ensure their training data has fair and accurate representations of diversity. Cramer et al. [\u003cspan citationid=\"CR102\" class=\"CitationRef\"\u003e102\u003c/span\u003e] present helpful checklists for AI developers to help them think critically about diversity and representation by considering the purpose of the system, the output of the system and its impact on society and users before they begin developing it. However, a significant challenge in addressing these issues lies in the lack of algorithmic transparency on social media platforms. The opaque nature of these algorithms creates challenges for researchers and policymakers to assess what data is being used in training and how it influences content curation and recommendations [\u003cspan citationid=\"CR103\" class=\"CitationRef\"\u003e103\u003c/span\u003e, \u003cspan citationid=\"CR104\" class=\"CitationRef\"\u003e104\u003c/span\u003e]. This hinders efforts to identify and correct the biases. Despite this, it remains essential for developers and platforms to be mindful of these concerns and proactively adopt inclusive practices such as sociotechnical transparency frameworks [\u003cspan citationid=\"CR103\" class=\"CitationRef\"\u003e103\u003c/span\u003e] to create socially responsible AI systems.\u003c/p\u003e\u003cp\u003eRecent work by Soubutts et al. [\u003cspan citationid=\"CR105\" class=\"CitationRef\"\u003e105\u003c/span\u003e] on digital mental health service design with culturally diverse young people presented a call to action for researchers to actively consider the engagement of culturally diverse people in technology design. During technological development, it is important to ensure that solutions resonate with individuals' needs, requirements and experiences. This is even more important in body image and ED, with significant gender and cultural biases in its literature base [\u003cspan citationid=\"CR100\" class=\"CitationRef\"\u003e100\u003c/span\u003e, \u003cspan citationid=\"CR106\" class=\"CitationRef\"\u003e106\u003c/span\u003e].\u003c/p\u003e\u003cp\u003e\u003cb\u003eUnderstanding the context of harmful content\u003c/b\u003e\u003c/p\u003e\u003cp\u003eOur findings indicated there is a need to create a robust set of rules for what is considered harmful and safe for people at-risk or with ED. This classification can be utilised to train algorithms to perform better content moderation, moving the field away from a reliance on hashtags and keywords. Evading hashtag moderation by using mutated hashtags, avoiding hashtags altogether, or hijacking and using hashtags meant for different content (e.g. body positive social media content) has been a go-to method for specific content creators and advertisers to keep their harmful content online [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. However, before we can delve fully into the classification of these content, it is essential to understand the context surrounding certain content to ascertain whether it is harmful or not.\u003c/p\u003e\u003cp\u003eOur findings showed that the intent of certain content is a key factor in determining its level of potential harm. Prior research has explored the use of AI technologies to generate captions and descriptions for image and video content by understanding and exploring the content's context [\u003cspan citationid=\"CR107\" class=\"CitationRef\"\u003e107\u003c/span\u003e, \u003cspan citationid=\"CR108\" class=\"CitationRef\"\u003e108\u003c/span\u003e]. This research can be leveraged in future work surrounding image-based social media classification by bolstering this generic contextual understanding with rules for classifying harmful content. For example, model alignment (a process of embedding human values into AI models and manipulating its output to align with human goals and rules) [\u003cspan citationid=\"CR109\" class=\"CitationRef\"\u003e109\u003c/span\u003e] within generative AI systems that can provide textual descriptions of videos and images (e.g. videoLlama, chatGPT). The implementation of highly robust methodologies used in healthcare research, such as the Delphi technique [\u003cspan citationid=\"CR110\" class=\"CitationRef\"\u003e110\u003c/span\u003e], which is a systematic process for gathering consensus, could be used in future research to drive the development of rules that are highly regarded as relevant and necessary for future AI systems aiming to classify harmful content.\u003c/p\u003e\u003cp\u003eWe propose that a useful direction for future research would be to develop AI systems that could be used by content creators. Rather than focusing solely on moderating harmful content after it appears on social media, it would be more effective and practical to prevent such content from being created and uploaded in the first place. In line with this preventative strategy, the Butterfly Foundation, in partnership with Instagram, has launched a social media series featuring Australian content creators who share their experiences with mindful content creation to avoid unintentional impact on body image perceptions, with a focus on promoting the well-being of their audiences [\u003cspan citationid=\"CR111\" class=\"CitationRef\"\u003e111\u003c/span\u003e]. This highlights the importance of educating content creators to be more aware and mindful of how their content can influence viewers, fostering a more responsible and supportive digital environment.\u003c/p\u003e\u003cp\u003e\u003cb\u003eLimitations\u003c/b\u003e\u003c/p\u003e\u003cp\u003e​​Despite recruitment efforts to recruit participants from diverse backgrounds for expert interviews and focus groups with individuals with lived experience, most of our participants were White and female. In addition, the majority of our lived experience participants had experiences of anorexia, meaning that the experiences of participants with bulimia, binge eating disorder and other ED classifications were not fully represented in our sample. While our research had a diversity level in its sample, our numbers were small. While a challenging task, it is vital that future research attempts to amplify the voices of these traditionally underrepresented voices in ED research. Approaching multiple lived experience networks, or recruiting directly from formal ED services, particularly those that focus on more marginalised experiences of ED, might aid more diversity in recruitment samples in the future. We did not collect data on participants' sexual orientation in this study. However, this could be considered in future research to provide a more comprehensive understanding.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eOur paper reports on the range of contexts of potentially harmful content in social media for individuals at-risk of, or experiencing ED. The understanding of these contexts aims to empirically inform the future classification efforts of such content, and the role of technology in supporting this process. We have presented unique insights into the nuances and subjectivity of such content, and the factors that could tip the balance between harmful, ambiguous and safe content. This paper also presents potential technological solutions for future research to mitigate harmful social media content. However, challenges remain in developing technological solutions that safeguard against harmful content, while maintaining balance with 'freedom of speech' and avoiding censorship. We also emphasise the need to enhance the diverse representations in social media, by focusing on the promotion of non-appearance-related and diversified content regarding body shape, size, gender, body functionality and cultural backgrounds. Finally, we have provided a set of recommendations for the design of future technological solutions to reduce the negative impact of social media by supporting different approaches to content moderation.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eED\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eEating disorders\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eAI\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eArtificial intelligence\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was approved by Monash University Human Research Ethics Committee, Monash University under the approval number ID- 41131. All participants provided informed consent prior to the participation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe participants provided informed consent for the publication of research data and findings, with the understanding that no identifying information would be disclosed in any publications.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data collected and analysed during the study are not publicly available due to the participant confidentiality requirements outlined in the ethics approval granted by the Monash University Human Research Ethics Committee.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding declaration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo funding to be declared.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePS led the original draft, methodology, data collection, data analysis, and conceptualisation. JX, PDH, and RM contributed to the writing through review and editing and were involved in supervision and conceptualisation. MB supported the work through review and editing and supervision, while SG contributed to writing (review and editing) and provided support and resources for data collection. RM also played a key role in data analysis.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003evan Hoeken D, Hoek HW. Review of the burden of eating disorders: mortality, disability, costs, quality of life, and family burden. Curr Opin Psychiatry LWW; 2020;33(6):521\u0026ndash;527.\u003c/li\u003e\n \u003cli\u003eGorrell S, Murray SB. Eating Disorders in Males. Child Adolesc Psychiatr Clin N Am 2019;28(4):641\u0026ndash;651. doi: https://doi.org/10.1016/j.chc.2019.05.012\u003c/li\u003e\n \u003cli\u003eMallaram GK, Sharma P, Kattula D, Singh S, Pavuluru P. Body image perception, eating disorder behavior, self-esteem and quality of life: a cross-sectional study among female medical students. J Eat Disord Springer; 2023;11(1):225.\u003c/li\u003e\n \u003cli\u003eCastellano S, Rizzotto A, Neri S, Currenti W, Guerrera CS, Pirrone C, Coco M, Corrado DD. The Relationship between Body Dissatisfaction and Eating Disorder Symptoms in Young Women Aspiring Fashion Models: The Mediating Role of Stress. Eur J Investig Health Psychol Educ 2021;11(2):607\u0026ndash;615. doi: https://doi.org/10.3390/ejihpe11020043\u003c/li\u003e\n \u003cli\u003eRodgers RF, Melioli T. The relationship between body image concerns, eating disorders and internet use, part I: A review of empirical support. Adolesc Res Rev Springer; 2016;1:95\u0026ndash;119.\u003c/li\u003e\n \u003cli\u003eLeonard KC. The Impact of Social Media Body Challenges on Youths\u0026rsquo; Body Image [Master\u0026rsquo;s Thesis]. North Dakota State University; 2020.\u003c/li\u003e\n \u003cli\u003eHan SH, Safeek R, Ockerman K, Trieu N, Mars P, Klenke A, Furnas H, Sorice-Virk S. Public Interest in the Off-Label Use of Glucagon-like Peptide 1 Agonists (Ozempic) for Cosmetic Weight Loss: A Google Trends Analysis. Aesthet Surg J 2023 Dec 14;44(1):60\u0026ndash;67. doi: 10.1093/asj/sjad211\u003c/li\u003e\n \u003cli\u003eGriffiths S, Castle D, Cunningham M, Murray SB, Bastian B, Barlow FK. How does exposure to thinspiration and fitspiration relate to symptom severity among individuals with eating disorders? Evaluation of a proposed model. Body Image Elsevier; 2018;27:187\u0026ndash;195.\u003c/li\u003e\n \u003cli\u003eMahon C, Hevey D. Processing body image on social media: Gender differences in adolescent boys\u0026rsquo; and girls\u0026rsquo; agency and active coping. Front Psychol Frontiers Media SA; 2021;12:626763.\u003c/li\u003e\n \u003cli\u003eMoorman EL, Warnick JL, Acharya R, Janicke DM. The use of internet sources for nutritional information is linked to weight perception and disordered eating in young adolescents. Appetite Elsevier; 2020;154.\u003c/li\u003e\n \u003cli\u003eSanzari CM, Gorrell S, Anderson LM, Reilly EE, Niemiec MA, Orloff NC, Anderson DA, Hormes JM. The impact of social media use on body image and disordered eating behaviors: Content matters more than duration of exposure. Eat Behav Elsevier; 2023;49:101722.\u003c/li\u003e\n \u003cli\u003eWu Y, Kemps E, Prichard I. Digging into digital buffets: A systematic review of eating-related social media content and its relationship with body image and eating behaviours. Body Image Elsevier; 2024;48:101650.\u003c/li\u003e\n \u003cli\u003eSpence R, Bifulco A, Bradbury P, Martellozzo E, DeMarco J. The psychological impacts of content moderation on content moderators: A qualitative study. Cyberpsychology J Psychosoc Res Cyberspace 2023 Sep 18;17(4). doi: 10.5817/CP2023-4-8\u003c/li\u003e\n \u003cli\u003eGillespie T. Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press; 2019. doi: 10.12987/9780300235029ISBN:978-0-300-23502-9\u003c/li\u003e\n \u003cli\u003eHow We\u0026rsquo;re Supporting People Affected by Eating Disorders and Negative Body Image. Meta; 2021. doi: https://about.fb.com/news/2021/02/supporting-people-affected-by-eating-disorders-and-negative-body-image/\u003c/li\u003e\n \u003cli\u003eHelp Center. Instagram; 2024. doi: https://help.instagram.com/252214974954612\u003c/li\u003e\n \u003cli\u003eSafety Center. TikTok; 2024. doi: https://www.tiktok.com/safety/en/eating-disorder\u003c/li\u003e\n \u003cli\u003ePeebles R, Wilson JL, Litt IF, Hardy KK, Lock JD, Mann JR, Borzekowski DL. Disordered Eating in a Digital Age: Eating Behaviors, Health, and Quality of Life in Users of Websites With Pro-Eating Disorder Content. J Med INTERNET Res.\u003c/li\u003e\n \u003cli\u003e\u0026lsquo;Thinstagram\u0026rsquo;: Instagram\u0026rsquo;s algorithm fuels eating disorder epidemic. Tech Transparency Project; 2021. doi: https://www.techtransparencyproject.org/articles/thinstagram-instagrams-algorithm-fuels-eating-disorder-epidemic\u003c/li\u003e\n \u003cli\u003eGerrard Y. Beyond the hashtag: Circumventing content moderation on social media. New Media Soc SAGE Publications Sage UK: London, England; 2018;20(12):4492\u0026ndash;4511.\u003c/li\u003e\n \u003cli\u003eDrivas M, Reed OS, Berndt-Goke M. #WhatIEatInADay: The effects of viewing food diary TikTok videos on young adults\u0026rsquo; body image and intent to diet. Body Image 2024 Jun;49:101712. doi: 10.1016/j.bodyim.2024.101712\u003c/li\u003e\n \u003cli\u003eDavis HA, Patarinski AGG, Kosmas JA, Roske C, Kells M. Does watching short-form #WhatIEatInADay videos impact eating disorder cognitions and urges to engage in eating disorder behaviors? An experimental investigation of TikTok. Appetite 2025 Jul;211:108002. doi: 10.1016/j.appet.2025.108002\u003c/li\u003e\n \u003cli\u003eGagrčin E. Your social ties, your personal public sphere, your responsibility: How users construe a sense of personal responsibility for intervention against uncivil comments on Facebook. New Media Soc 2024 Aug;26(8):4299\u0026ndash;4316. doi: 10.1177/14614448221117499\u003c/li\u003e\n \u003cli\u003eKaňkov\u0026aacute; J, Binder A, Matthes J. Helpful or harmful? Navigating the impact of social media influencers\u0026rsquo; health advice: insights from health expert content creators. BMC Public Health 2024 Dec 18;24(1):3511. doi: 10.1186/s12889-024-21095-3\u003c/li\u003e\n \u003cli\u003eHaime Z, Kennedy L, Grace L, Biddle L. Experiences of Moderation, Moderators, and Moderating by Online Users Who Engage with Self-Harm and Suicide Content. Digit Soc 2025 Apr;4(1):8. doi: 10.1007/s44206-025-00166-x\u003c/li\u003e\n \u003cli\u003eGongane VU, Munot MV, Anuse AD. Detection and moderation of detrimental content on social media platforms: current status and future directions. Soc Netw Anal Min 2022 Dec;12(1):129. doi: 10.1007/s13278-022-00951-3\u003c/li\u003e\n \u003cli\u003eYan H, Fitzsimmons-Craft EE, Goodman M, Krauss M, Das S, Cavazos-Rehg P. Automatic detection of eating disorder-related social media posts that could benefit from a mental health intervention. Int J Eat Disord Wiley Online Library; 2019;52(10):1150\u0026ndash;1156.\u003c/li\u003e\n \u003cli\u003eRam\u0026iacute;rez-Cifuentes D, Mayans M, Freire A. Early risk detection of anorexia on social media. Internet Sci 5th Int Conf INSCI 2018 St Petersburg Russ Oct 24\u0026ndash;26 2018 Proc 5 Springer; 2018. p. 3\u0026ndash;14.\u003c/li\u003e\n \u003cli\u003eAbuhassan M, Anwar T, Liu C, Jarman HK, Fuller-Tyszkiewicz M. EDNet: Attention-Based Multimodal Representation for Classification of Twitter Users Related to Eating Disorders. Proc ACM Web Conf 2023 2023. p. 4065\u0026ndash;4074.\u003c/li\u003e\n \u003cli\u003ePark J, Gracie J, Alsoubai A, Stringhini G, Singh V, Wisniewski P. Towards Automated Detection of Risky Images Shared by Youth on Social Media. Companion Proc ACM Web Conf 2023 Austin TX USA: ACM; 2023. p. 1348\u0026ndash;1357. doi: 10.1145/3543873.3587607\u003c/li\u003e\n \u003cli\u003eDey AK. Understanding and Using Context. Pers Ubiquitous Comput 2001;\u003c/li\u003e\n \u003cli\u003eHow does Facebook use artificial intelligence to moderate content? Meta; 2024. doi: https://www.facebook.com/help/1584908458516247\u003c/li\u003e\n \u003cli\u003eSetting the rules for a safe TikTok experience. TikTok; 2024. doi: https://www.tiktok.com/transparency/en-gb/content-moderation/\u003c/li\u003e\n \u003cli\u003eHow Instagram uses artificial intelligence to moderate content. Instagram; 2024. doi: https://help.instagram.com/423837189385631\u003c/li\u003e\n \u003cli\u003eShirky C. The political power of social media: Technology, the public sphere, and political change. Foreign Aff JSTOR; 2011;28\u0026ndash;41.\u003c/li\u003e\n \u003cli\u003eFacebook Community Standards. Meta; 2024. doi: https://transparency.meta.com/en-gb/policies/community-standards/?source=https%3A%2F%2Fwww.facebook.com%2Fcommunitystandards\u003c/li\u003e\n \u003cli\u003eHall R, Wilmot C. Meta faces Ghana lawsuits over impact of extreme content on moderators. The Guardian 2025 Apr 27; Available from: https://www.theguardian.com/technology/2025/apr/27/meta-faces-ghana-lawsuits-over-impact-of-extreme-content-on-moderators\u003c/li\u003e\n \u003cli\u003eCard C. How Facebook AI Helps Suicide Prevention. Meta;\u003c/li\u003e\n \u003cli\u003eMeta M. Suicide prevention. Available from: https://about.meta.com/actions/safety/topics/wellbeing/suicideprevention\u003c/li\u003e\n \u003cli\u003eHelp Centre. If you see a post on Facebook that suggests someone has an eating disorder. Available from: https://www.facebook.com/help/230070543674737/\u003c/li\u003e\n \u003cli\u003eEating-disorder patients confront Meta bosses at federal parliament as research highlights TikTok\u0026rsquo;s negative influence. ABC News; 2023. doi: https://www.abc.net.au/news/2023-09-14/eating-disorder-patients-to-confront-meta-bosses-at-parliament/102847320\u003c/li\u003e\n \u003cli\u003eGriffiths S, Harris EA, Whitehead G, Angelopoulos F, Stone B, Grey W, Dennis S. Does TikTok contribute to eating disorders? A comparison of the TikTok algorithms belonging to individuals with eating disorders versus healthy controls. Body Image 2024 Dec;51:101807. doi: 10.1016/j.bodyim.2024.101807\u003c/li\u003e\n \u003cli\u003eYouTube\u0026rsquo;s Anorexia Algorithm How YouTube recommends eating disorders videos to young girls. Center for Countering Digital Hate; 2024. Available from: https://counterhate.com/wp-content/uploads/2024/12/CCDH.YoutubeED.Nov24.Report_FINAL.pdf\u003c/li\u003e\n \u003cli\u003eRodgers RF, McLean SA, Paxton SJ. Enhancing understanding of social media literacy to better inform prevention of body image and eating disorders. Eat Disord Taylor \u0026amp; Francis; 2024;1\u0026ndash;19.\u003c/li\u003e\n \u003cli\u003eJhaver S, Zhang AQ, Chen Q, Natarajan N, Wang R, Zhang A. Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor. arXiv; 2023. doi: 10.48550/arXiv.2305.10374\u003c/li\u003e\n \u003cli\u003eHolland G, Tiggemann M. A systematic review of the impact of the use of social networking sites on body image and disordered eating outcomes. Body Image Elsevier; 2016;17:100\u0026ndash;110.\u003c/li\u003e\n \u003cli\u003eJiotsa B, Naccache B, Duval M, Rocher B, Grall-Bronnec M. Social media use and body image disorders: Association between frequency of comparing one\u0026rsquo;s own physical appearance to that of people being followed on social media and body dissatisfaction and drive for thinness. Int J Environ Res Public Health MDPI; 2021;18(6):2880.\u003c/li\u003e\n \u003cli\u003eCasale S, Gemelli G, Calosi C, Giangrasso B, Fioravanti G. Multiple exposure to appearance-focused real accounts on Instagram: Effects on body image among both genders. Curr Psychol Springer; 2021;40:2877\u0026ndash;2886.\u003c/li\u003e\n \u003cli\u003ePedalino F, Camerini A-L. Instagram use and body dissatisfaction: The mediating role of upward social comparison with peers and influencers among young females. Int J Environ Res Public Health MDPI; 2022;19(3):1543.\u003c/li\u003e\n \u003cli\u003eMcLean SA, Wertheim EH, Masters J, Paxton SJ. A pilot evaluation of a social media literacy intervention to reduce risk factors for eating disorders. Int J Eat Disord Wiley Online Library; 2017;50(7):847\u0026ndash;851.\u003c/li\u003e\n \u003cli\u003eRodgers RF, Wertheim EH, Paxton SJ, Tylka TL, Harriger JA. # Bopo: Enhancing body image through body positive social media-evidence to date and research directions. Body Image Elsevier; 2022;41:367\u0026ndash;374.\u003c/li\u003e\n \u003cli\u003eTylka TL, Wood-Barcalow NL. What is and what is not positive body image? Conceptual foundations and construct definition. Body Image Elsevier; 2015;14:118\u0026ndash;129.\u003c/li\u003e\n \u003cli\u003eMcLean SA, Wertheim EH, Marques MD, Paxton SJ. Dismantling prevention: Comparison of outcomes following media literacy and appearance comparison modules in a randomised controlled trial. J Health Psychol Sage Publications Sage UK: London, England; 2019;24(6):761\u0026ndash;776.\u003c/li\u003e\n \u003cli\u003eSocial media literacy. SMRI; 2024. doi: https://www.smri.world/dictionary/social-media-literacy\u003c/li\u003e\n \u003cli\u003eWood-Barcalow NL, Tylka TL, Augustus-Horvath CL. \u0026ldquo;But I like my body\u0026rdquo;: Positive body image characteristics and a holistic model for young-adult women. Body Image Elsevier; 2010;7(2):106\u0026ndash;116.\u003c/li\u003e\n \u003cli\u003eAndrew R, Tiggemann M, Clark L. Predictors and health-related outcomes of positive body image in adolescent girls: A prospective study. Dev Psychol American Psychological Association; 2016;52(3):463.\u003c/li\u003e\n \u003cli\u003eCohen R, Irwin L, Newton-John T, Slater A. # bodypositivity: A content analysis of body positive accounts on Instagram. Body Image Elsevier; 2019;29:47\u0026ndash;57.\u003c/li\u003e\n \u003cli\u003eWebb JB, Vinoski ER, Bonar AS, Davies AE, Etzel L. Fat is fashionable and fit: A comparative content analysis of Fatspiration and Health at Every Size\u0026reg; Instagram images. Body Image Elsevier; 2017;22:53\u0026ndash;64.\u003c/li\u003e\n \u003cli\u003eBrino KAS, Derouin AL, Silva SG. Problematic internet use in adolescents and implementation of a social media hygiene protocol. J Pediatr Nurs Elsevier; 2022;63:84\u0026ndash;89.\u003c/li\u003e\n \u003cli\u003eHaime Z, Kennedy L, Grace L, Cohen R, Derges J, Biddle L. The Journey of Engaging With Web-Based Self-Harm and Suicide Content: Longitudinal Qualitative Study. JMIR Infodemiology 2024 Mar 28;4:e47699. doi: 10.2196/47699\u003c/li\u003e\n \u003cli\u003eCarey M. The Affects Social Media has on People with Eating Disorders. Ir Assoc Couns Psychother 2015;15(4).\u003c/li\u003e\n \u003cli\u003eKhurana D, Koli A, Khatter K, Singh S. Natural language processing: state of the art, current trends and challenges. Multimed Tools Appl Springer; 2023;82(3):3713\u0026ndash;3744.\u003c/li\u003e\n \u003cli\u003eBen\u0026iacute;tez-Andrades JA, Alija-P\u0026eacute;rez JM, Garc\u0026iacute;a-Rodr\u0026iacute;guez I, Benavides C, Alaiz-Moret\u0026oacute;n H, Vargas RP, Garc\u0026iacute;a-Ord\u0026aacute;s MT. BERT Model-Based Approach For Detecting Categories of Tweets in the Field of Eating Disorders (ED). 2021 IEEE 34th Int Symp Comput-Based Med Syst CBMS IEEE; 2021. p. 586\u0026ndash;590.\u003c/li\u003e\n \u003cli\u003eFeldman J. Classifying Pro-Eating Disorder Images on Twitter Using Deep Learning. Intersect Stanf J Sci Technol Soc 2023;16(2).\u003c/li\u003e\n \u003cli\u003eKhan AA, Laghari AA, Awan SA. Machine learning in computer vision: a review. EAI Endorsed Trans Scalable Inf Syst 2021;8(32):e4\u0026ndash;e4.\u003c/li\u003e\n \u003cli\u003eChancellor S, Pater JA, Clear T, Gilbert E, De Choudhury M. #thyghgapp: Instagram Content Moderation and Lexical Variation in Pro-Eating Disorder Communities. Proc 19th ACM Conf Comput-Support Coop Work Soc Comput 2016. p. 1201\u0026ndash;1213.\u003c/li\u003e\n \u003cli\u003eLlans\u0026oacute; E, van Hoboken J, Leerssen P, Harambam J. Artificial Intelligence, Content Moderation, and Freedom of Expression.\u003c/li\u003e\n \u003cli\u003eHosseini H, Kannan S, Zhang B, Poovendran R. Deceiving Google\u0026rsquo;s Perspective API Built for Detecting Toxic Comments. arXiv; 2017. doi: 10.48550/arXiv.1702.08138\u003c/li\u003e\n \u003cli\u003eLees A, Tran VQ, Tay Y, Sorensen J, Gupta J, Metzler D, Vasserman L. A New Generation of Perspective API: Efficient Multilingual Character-level Transformers. arXiv; 2022. doi: 10.48550/arXiv.2202.11176\u003c/li\u003e\n \u003cli\u003eCavus N, Goksu M, Oktekin B. Real-time fake news detection in online social networks: FANDC Cloud-based system. Sci Rep 2024 Oct 29;14(1):25954. doi: 10.1038/s41598-024-76102-9\u003c/li\u003e\n \u003cli\u003eMaham S, Tariq A, Khan MUG, Alamri FS, Rehman A, Saba T. ANN: adversarial news net for robust fake news classification. Sci Rep 2024 Apr 3;14(1):7897. doi: 10.1038/s41598-024-56567-4\u003c/li\u003e\n \u003cli\u003eZhang C, Gupta A, Qin X, Zhou Y. A computational approach for real-time detection of fake news. Expert Syst Appl 2023 Jul;221:119656. doi: 10.1016/j.eswa.2023.119656\u003c/li\u003e\n \u003cli\u003eKhan SA, Sheikhi G, Opdahl AL, Rabbi F, Stoppel S, Trattner C, Dang-Nguyen D-T. Visual User-Generated Content Verification in Journalism: An Overview. IEEE Access 2023;11:6748\u0026ndash;6769. doi: 10.1109/ACCESS.2023.3236993\u003c/li\u003e\n \u003cli\u003eUrbani S. Verifying Online Information. First Draft; Available from: https://firstdraftnews.org/long-form-article/verifying-online-information/\u003c/li\u003e\n \u003cli\u003eBacha J, Ullah F, Khan J, Sardar AW, Lee S. A Deep Learning-Based Framework for Offensive Text Detection in Unstructured Data for Heterogeneous Social Media. IEEE Access 2023;11:124484\u0026ndash;124498. doi: 10.1109/ACCESS.2023.3330081\u003c/li\u003e\n \u003cli\u003eSuicide, self-injury and eating disorders. Meta; Available from: https://transparency.meta.com/en-gb/policies/community-standards/suicide-self-injury/\u003c/li\u003e\n \u003cli\u003eArora A, Nakov P, Hardalov M, Sarwar SM, Nayak V, Dinkov Y, Zlatkova D, Dent K, Bhatawdekar A, Bouchard G, Augenstein I. Detecting Harmful Content on Online Platforms: What Platforms Need vs. Where Research Efforts Go. ACM Comput Surv 2024 Mar 31;56(3):1\u0026ndash;17. doi: 10.1145/3603399\u003c/li\u003e\n \u003cli\u003eBraun V, Clarke V. Successful Qualitative Research a practical guide for beginners.\u003c/li\u003e\n \u003cli\u003eProposed Social Media Ban for Under-16s in Australia. Aust Hum Rights Comm 2024 Nov 21; Available from: https://humanrights.gov.au/about/news/proposed-social-media-ban-under-16s-australia\u003c/li\u003e\n \u003cli\u003eMcNaney R, Vines J, Mercer J, Mexter L, Welsh D, Young T. DemYouth: Co-designing and enacting tools to support young people\u0026rsquo;s engagement with people with dementia. Proc 2017 CHI Conf Hum Factors Comput Syst New York, NY, USA: ACM; 2017. p. 1313\u0026ndash;1325. doi: https://doi.org/10.1145/3025453.3025558\u003c/li\u003e\n \u003cli\u003eBriggs P, Blythe M, Vines J, Lindsay S, Dunphy P, Nicholson J, Green D, Kitson J, Monk A, Olivier P. Invisible design: exploring insights and ideas through ambiguous film scenarios. Proc Des Interact Syst Conf Newcastle Upon Tyne United Kingdom: ACM; 2012. p. 534\u0026ndash;543. doi: 10.1145/2317956.2318036\u003c/li\u003e\n \u003cli\u003eBraun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3(2):77\u0026ndash;101.\u003c/li\u003e\n \u003cli\u003eCarter N, Bryant-Lukosius D, DiCenso A, Blythe J, Neville AJ. The Use of Triangulation in Qualitative Research. Oncol Nurs Forum 2014 Sep 1;41(5):545\u0026ndash;547. doi: 10.1188/14.ONF.545-547\u003c/li\u003e\n \u003cli\u003eMichael Nnaemeka Ajemba, Ebube Chinwe Arene. Possible advantages that may be enhanced with the adoption of research triangulation or mixed methodology. Magna Sci Adv Res Rev 2022 Sep 30;6(1):058\u0026ndash;061. doi: 10.30574/msarr.2022.6.1.0066\u003c/li\u003e\n \u003cli\u003eHockin-Boyers H, Pope S, Jamie K. Digital pruning: Agency and social media use as a personal political project among female weightlifters in recovery from eating disorders. New Media Soc SAGE Publications Sage UK: London, England; 2021;23(8):2345\u0026ndash;2366.\u003c/li\u003e\n \u003cli\u003eKhursheed A. Role of Media Literacy in Spreading Body Positivity and in Preventing Eating Disorder. Int J Indian Psychȯlogy 2023;11(1).\u003c/li\u003e\n \u003cli\u003eFardouly J, Slater A, Parnell J, Diedrichs PC. Can following body positive or appearance neutral Facebook pages improve young women\u0026rsquo;s body image and mood? Testing novel social media micro-interventions. Body Image Elsevier; 2023;44:136\u0026ndash;147.\u003c/li\u003e\n \u003cli\u003eLewis-Smith H, Garbett KM, Chaudhry A, Dhillon M, Shroff H, White P, Diedrichs PC. Evaluating a body image school-based intervention in India: A randomized controlled trial. Body Image Elsevier; 2023;44:148\u0026ndash;156.\u003c/li\u003e\n \u003cli\u003eGhaznavi J, Taylor LD. Bones, body parts, and sex appeal: An analysis of# thinspiration images on popular social media. Body Image Elsevier; 2015;14:54\u0026ndash;61.\u003c/li\u003e\n \u003cli\u003eTiggemann M, Churches O, Mitchell L, Brown Z. Tweeting weight loss: A comparison of# thinspiration and# fitspiration communities on Twitter. Body Image Elsevier; 2018;25:133\u0026ndash;138.\u003c/li\u003e\n \u003cli\u003eVega J, Bell BT, Taylor C, Xie J, Ng H, Honary M, McNaney R. Detecting mental health behaviors using mobile interactions: Exploratory study focusing on binge eating. JMIR Ment Health JMIR Publications Toronto, Canada; 2022;9(4):e32146.\u003c/li\u003e\n \u003cli\u003eKozyreva A, Herzog SM, Lewandowsky S, Hertwig R, Lorenz-Spreen P, Leiser M, Reifler J. Resolving content moderation dilemmas between free speech and harmful misinformation. Proc Natl Acad Sci National Acad Sciences; 2023;120(7):e2210666120.\u003c/li\u003e\n \u003cli\u003eHerrick SSC, Hallward L, Duncan LR. \u0026ldquo;This is just how I cope\u0026rdquo;: An inductive thematic analysis of eating disorder recovery content created and shared on TikTok using # EDrecovery. Int J Eat Disord 2021 Apr;54(4):516\u0026ndash;526. doi: 10.1002/eat.23463\u003c/li\u003e\n \u003cli\u003eNot just algorithms: assuring user safety online with systemic regulatory frameworks. Reset Australia; 2024. doi: https://apo.org.au/node/326122\u003c/li\u003e\n \u003cli\u003eAu ES, Cosh SM. Social media and eating disorder recovery: An exploration of Instagram recovery community users and their reasons for engagement. Eat Behav Elsevier; 2022;46:101651.\u003c/li\u003e\n \u003cli\u003eRong EZ, Zhou MM, Lu Z, Fan M. \u0026ldquo;It Feels Like Being Locked in A Cage\u0026rdquo;: Understanding Blind or Low Vision Streamers\u0026rsquo; Perceptions of Content Curation Algorithms. Proc 2022 ACM Des Interact Syst Conf 2022. p. 571\u0026ndash;585.\u003c/li\u003e\n \u003cli\u003eBiddle S, Riberio PV, Dias T. Invisible Censorship: TikTok Told Moderators to Suppress Posts by \u0026ldquo;Ugly\u0026rdquo; People and the Poor to Attract New Users. The Intercept; 2020. doi: https://theintercept.com/2020/03/16/tiktok-app-moderatorsusers-discrimination/\u003c/li\u003e\n \u003cli\u003eKarizat N, Delmonaco D, Eslami M, Andalibi N. Algorithmic folk theories and identity: How TikTok users co-produce Knowledge of identity and engage in algorithmic resistance. Proc ACM Hum-Comput Interact ACM New York, NY, USA; 2021;5(CSCW2):1\u0026ndash;44.\u003c/li\u003e\n \u003cli\u003eSimpson E, Semaan B. For you, or for\u0026quot; you\u0026quot;? Everyday LGBTQ+ encounters with TikTok. Proc ACM Hum-Comput Interact ACM New York, NY, USA; 2021;4(CSCW3):1\u0026ndash;34.\u003c/li\u003e\n \u003cli\u003eShrestha P, Delir Haghighi P, Sharp G, Xie J, McNaney R. One Size (Doesn\u0026rsquo;t) Fit All: Exploring design considerations for digital body dissatisfaction interventions with underrepresented populations. Proc 2024 ACM Des Interact Syst Conf 2024. p. 3534\u0026ndash;3550.\u003c/li\u003e\n \u003cli\u003eWang T, Zhao J, Yatskar M, Chang K-W, Ordonez V. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. Proc IEEECVF Int Conf Comput Vis 2019. p. 5310\u0026ndash;5319.\u003c/li\u003e\n \u003cli\u003eCramer H, Garcia-Gathright J, Springer A, Reddy S. Assessing and addressing algorithmic bias in practice. Interactions ACM New York, NY, USA; 2018;25(6):58\u0026ndash;63.\u003c/li\u003e\n \u003cli\u003eGausen A, Guo C, Luk W. An approach to sociotechnical transparency of social media algorithms using agent-based modelling. AI Ethics 2025 Apr;5(2):1827\u0026ndash;1845. doi: 10.1007/s43681-024-00527-1\u003c/li\u003e\n \u003cli\u003eExploring social media users\u0026rsquo; experiences with algorithmic transparency cues. New Media.\u003c/li\u003e\n \u003cli\u003eSoubutts E, Shrestha P, Davidson BI, Qu C, Mindel C, Sefi A, Marshall P, McNaney R. Challenges and Opportunities for the Design of Inclusive Digital Mental Health Tools: Understanding Culturally Diverse Young People\u0026rsquo;s Experiences. 2024. p. 1\u0026ndash;16.\u003c/li\u003e\n \u003cli\u003eBorowsky HM, Eisenberg ME, Bucchianeri MM, Piran N, Neumark-Sztainer D. Feminist identity, body image, and disordered eating. Eat Disord Taylor \u0026amp; Francis; 2016;24(4):297\u0026ndash;311.\u003c/li\u003e\n \u003cli\u003eGao L, Guo Z, Zhang H, Xu X, Shen HT. Video captioning with attention-based LSTM and semantic consistency. IEEE Trans Multimed IEEE; 2017;19(9):2045\u0026ndash;2055.\u003c/li\u003e\n \u003cli\u003eHan T, Bain M, Nagrani A, Varol G, Xie W, Zisserman A. AutoAD: Movie description in context. Proc IEEECVF Conf Comput Vis Pattern Recognit 2023. p. 18930\u0026ndash;18940.\u003c/li\u003e\n \u003cli\u003eShen T, Jin R, Huang Y, Liu C, Dong W, Guo Z, Wu X, Liu Y, Xiong D. Large language model alignment: A survey. ArXiv Prepr ArXiv230915025 2023;\u003c/li\u003e\n \u003cli\u003eNasa P, Jain R, Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol Baishideng Publishing Group Inc; 2021;11(4):116.\u003c/li\u003e\n \u003cli\u003ePause before you post: Social media \u0026lsquo;appearance-led\u0026rsquo; content impacting Australians\u0026rsquo; body image. Butterfly foundation; Available from: https://butterfly.org.au/news/pause-before-you-post-social-media-appearance-led-content-impacting-australians-body-image/\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":true,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-eating-disorders","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"joed","sideBox":"Learn more about [Journal of Eating Disorders](http://jeatdisord.biomedcentral.com)","snPcode":"40337","submissionUrl":"https://submission.nature.com/new-submission/40337/3","title":"Journal of Eating Disorders","twitterHandle":"@JEatDisord","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"social media, body dissatisfaction, eating disorders, social media content, content moderation, harmful content","lastPublishedDoi":"10.21203/rs.3.rs-7136130/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7136130/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe ubiquity of social media has increased exposure to idealised beauty standards, often unrealistic and harmful. Repeated exposure with such content has been linked to body dissatisfaction, harmful behaviours, and potentially the development of eating disorders (ED). Given the volume of content produced daily, effective harm mitigation strategies (automated or manually user-driven) are essential. Such strategies require empirically informed understanding of the underlying contexts and nuances surrounding harmful content.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eObjective:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study has two key aims: (1) to understand the perspectives of experts by profession and lived experience of eating disorders, on what makes social media content harmful in the context of body image and ED, including why and how this harm occurs; and (2) to explore how technology might help mitigate these effects.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe engaged n=30 participants in our work. We conducted 12 interviews with experts by profession (n=2 ED support service providers and n=10 body image and ED experts), and 5 focus groups with experts by lived experience (n=18 people with lived experience of ED).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe thematic analysis presented six prominent themes: (1) Understanding contextual factors of social media content, (2) Contributing factors to the ED \"echo chamber\", (3) Challenges for content moderation in social media, (4) Needs and requirements of stakeholders for a safer social media experience, (5) Promoting diversity on social media, and (6) Perceptions regarding use of technology to mitigate the negative impact of social media. Drawing on these insights, we developed a categorisation framework consisting of eight types of harmful social media content related to body image and ED This study provides an underlying contextual understanding of harmful content related to body image and ED and highlights essential considerations for harm-reducing technologies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eManual content safeguards and reporting place significant responsibility on users. Through this work, we present foundations for informed rules to differentiate between harmful, ambiguous, and safe content related to body image and ED, by highlighting the underlying context. We present design insights to inform how technology might support classification systems and dynamic, adaptable automated moderation, and key considerations for reducing social media harm.\u003c/p\u003e","manuscriptTitle":"Thigh Gaps and Filtered Snaps: A Qualitative Study Exploring Opportunities to Mitigate Social Media Harm Through Content Moderation for People with Eating Disorders","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-01 05:52:08","doi":"10.21203/rs.3.rs-7136130/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-11-17T18:45:05+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-16T20:21:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"62569215765995559506957048548832429449","date":"2025-10-30T19:46:26+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-25T19:25:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"1559952792863691158599469371096253710","date":"2025-07-25T15:55:03+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-25T14:58:42+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-18T11:13:50+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-18T11:12:20+00:00","index":"","fulltext":""},{"type":"submitted","content":"Journal of Eating Disorders","date":"2025-07-16T05:40:25+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-eating-disorders","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"joed","sideBox":"Learn more about [Journal of Eating Disorders](http://jeatdisord.biomedcentral.com)","snPcode":"40337","submissionUrl":"https://submission.nature.com/new-submission/40337/3","title":"Journal of Eating Disorders","twitterHandle":"@JEatDisord","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"82b1441a-9cef-462a-9e4a-b46b51e72568","owner":[],"postedDate":"September 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-01-26T16:01:24+00:00","versionOfRecord":{"articleIdentity":"rs-7136130","link":"https://doi.org/10.1186/s40337-025-01504-7","journal":{"identity":"journal-of-eating-disorders","isVorOnly":false,"title":"Journal of Eating Disorders"},"publishedOn":"2026-01-21 15:58:20","publishedOnDateReadable":"January 21st, 2026"},"versionCreatedAt":"2025-09-01 05:52:08","video":"","vorDoi":"10.1186/s40337-025-01504-7","vorDoiUrl":"https://doi.org/10.1186/s40337-025-01504-7","workflowStages":[]},"version":"v1","identity":"rs-7136130","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7136130","identity":"rs-7136130","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}