Consensus: How the sausage gets made

What can we learn about the experts involved in the recent social media effects consensus preprint and what does that say about the consensus.

Ruben C. Arslan https://rubenarslan.github.io (Psychological Research Methods, University Witten/Herdecke)
2025-05-20

My thanks to Valerio Capraro, the first author of the consensus statement, who gave me feedback on an earlier version of this post.

A large working group of psychologists recently put out a consensus statement on social media and mental health effects, based on Haidt’s book “The Anxious Generation”.

Many of the scholars I respect the most in this space (e.g., Lucy Foulkes, Amy Orben, Andrew Przybylski) seem to have not participated or dropped out of this effort. I know that in at least two cases this was because they did not trust the process. I am not an expert in this field myself, but I did read “The Anxious Generation”, “What Mental Illness Really Is … And What It Isn’t” and a few other things and have some familiarity with how strong the evidence base is. Now, personally, I think this consensus effort is premature. For many of the claims they examined we don’t have the evidence to assess many of these claims well. Consensus is not a shortcut to understanding the world.

In the end, the participants found consensus for only one causal claim (“Heavy smartphone and social media use can cause sleep problems.”), several weak claims about associations (e.g., “social media may be associated with visual social comparison in girls”) as well as a claim that adolescent mental health has declined in several Western countries. To me, not even the evidence for this mental health decline is very compelling. But generally this consensus is not far away from my own current understanding. What irks me is how this was communicated and what they chose to focus on.

It’s not your fault if recent social media posts led you to believe there was a “ton of consensus” that “Adolescent mental health has declined”1 or that rejecting this consensus is akin to denying climate change.2

A now deleted skeet by Jay van Bavel Screenshot of now deleted skeet by Jay van Bavel

However, even if I don’t think the consensus statement has much value, I think it’s meta-scientifically interesting to look at how the participating scholars assessed the level of evidence for each claim. Maybe this will also give us an idea what kind of experts self-selected into the process. Potentially, the Delphi method corrected for all problems. I am not trying to assess that. I am only looking at Study 1 here to understand “how the sausage gets made”. Similarly, I found it interesting to read how the first author dealt with critical comments, documented on OSF.

Naively, you might expect that sometimes, when people have expertise in one area but not another, they would not answer questions in those areas where they cannot assess the evidence base. You would not expect them to report a stronger evidence base than exists. Consider replicated experimental field studies. These are very effortful studies, done either at considerable expense or making use of widely reported policy changes. You certainly wouldn’t expect a researcher to say such studies exist in their field of expertise when they don’t. In fact, I fail my students when they make claims their citations cannot support (I’ve gotten stricter about this because it’s such an LLM hallmark). WELL!

You might be surprised to hear that, initially, for at least seven claims a substantial fraction of scholars said there were causally informative field studies yet none were cited in the final consensus statement’s evidence summary according to language model Gemini 2.5 Pro.3 The first author, Valerio Capraro, seems to agree that the evidence for most claims is weak and correlational. So, I don’t think my assessment that some scholars overestimated the evidence is out of whack with the first author’s views.

So, who are these scholars who think there have been replicated causally informative field studies showing that social media affects body image? Maybe they have references that didn’t make the cut. Unfortunately, I don’t know! Turns out the responses were anonymized, as is standard in the Delphi method. I.e. even for those scholars who participated non-anonymously, we don’t know what they said for each claim. Expertise without accountability? Truly, you can have it all.

What question did they answer exactly when assessing the evidence base for causal claims?

To the best of your knowledge, what is the level of evidence regarding this claim?

  • ▢ No evidence, only opinions, perspectives, general theory or anecdotes (1)
  • ▢ Some correlational evidence (laboratories, surveys, online, field) (2)
  • ▢ Some causal evidence but in limited settings (laboratories, surveys, and online, self-reported measures) (3)
  • ▢ Causal evidence in a field study (4)
  • ▢ Replicated causal evidence from field studies (5)
  • ▢ Wide-scale causal evidence from multiple field studies, policy evaluations or other natural settings (6)

What did they say in Study 1?

image

Figure caption for Figure 3

Figure 3. Summary of responses to the “level” question for each of the 26 claims from Survey 1, ordered as in Figure 2. For each claim, the “level” question asked in Survey 1 was: “To the best of your knowledge, what is the level of evidence regarding this claim?”. Available answers depended on the claim. For non-causal claims 1-7, 16, and 22-23, the available answers were: No evidence, only opinions, perspectives, general theory or anecdotes; Some empirical evidence but in limited settings (laboratories, surveys and online, self-reported measures); Field evidence; Replicated evidence in field studies or other natural settings; Wide-scale evidence from multiple field studies, policy evaluations or other natural settings. For causal claims 8-15, 17-21, and 24-26, the available answers were: No evidence, only opinions, perspectives, general theory or anecdotes; Some correlational evidence (laboratories, surveys, online, field); Some causal evidence but in limited settings (laboratories, surveys, and online, self-reported measures); Causal evidence in a field study; Replicated causal evidence from field studies; Wide-scale causal evidence from multiple field studies, policy evaluations or other natural settings. For visualization purposes, we shortened the wording of the claims. Refer to Table 1 for the exact wording of the claims.

As far as I could tell at a glance, the authors didn’t precisely define what counts as causal vs. correlational. I’m one of those weirdos who believe you can infer causality from observational/correlational data if you’re careful, but from the contrast to “correlational”, I guess the authors considered only experiments and natural experiments “causal evidence”.

Now, after examining this graph, you would be excused if you thought these categories were mutually exclusive. But no:

Moreover, after careful consideration within the core group, we decided to allow multiple answers for this question to avoid forcing participants to rank the level of evidence.

Yes, you certainly wouldn’t want to rank levels of evidence. How would those poor correlational studies and opinions feel? Still, you might be confused, after all, Figure 3 (above) tabulates them as if responses summed to 100%. But actually authors could and did endorse multiple mutually exclusive answers. Most obviously, more than 70 responses say both there is “no evidence” and at the same time say there is evidence. I guess they exist in a state of superposition. It would collapse if we actually observed the evidence, but, like, who has the time?

value n()
1,2 59
1,2,3 9
1,2,3,4,6 1
1,3 4
1,3,4 1
1,5 1
1,6 1

The paper didn’t document how the graph was made, so I asked) and it turns out they expanded the denominator. It sums to 100% because the denominator is “number of checkboxes clicked”. I don’t think that’s a useful denominator. People who checked no box are excluded and you can’t interpret the percentage as the number of experts who endorsed X.

I looked into this because several experts claimed that there were replicated causally informative field studies showing that social media increases visual social comparison in girls. That seemed odd, because a) this seems difficult to study experimentally in the field b) I looked at this literature briefly a while ago and didn’t encounter anything of the sort. So, I checked the evidence summary in Supplementary Section S5. Indeed, no field studies are highlighted at all. Weird. Did they imagine these studies? Do they exist, but for some reason the best-quality evidence (sorry, sorry, I know evidence shan’t be ranked) isn’t mentioned?

To find out how this stacks up for the other claims, I let Gemini 2.5 Pro take a stab at tabulating how often experts say studies exist that then seem to be missing in action in the evidence review. I only hand-checked the visual social comparison section. Gemini agreed with my assessment there. So, here we have it: It’s not just LLMs that occasionally hallucinate scientific papers.

Does this mean the consensus statement is wrong? They went through three more rounds and had a chance to rectify such problems with grounding. Still, I worry that people who admit uncertainty self-selected out and people who incorrectly believe causally informative field studies exist where there are none, self-selected in.

Claim % Causal Field (Est. from Fig 3) % Causal Field or Higher (Est. from Fig 3) S5 Text: Experimental Field, Natural Experiment, or Well-Controlled Design in Field Supporting Causal Claim? S5 Text: Observational Study Cited as Supporting Causal Claim?
8. Heavy daily use of smartphones and social media can cause sleep deprivation. ~10-12% ~25-30% Yes (Intervention studies like Martin et al., Perrault et al., Tu et al., described as improving sleep by reducing screen time, could be field experiments). Schechter et al. (meta-analysis of blue light interventions) and Hartstein et al. (consensus panel review) also point to experimental/interventional evidence. Yes (Longitudinal studies like Rod et al., Liu et al. cited for associations).
9. Chronic sleep deprivation can cause a decline in mental health. ~5-7% ~75-80% Yes (Scott et al. meta-analysis of sleep-improvement interventions showing mental health benefits is strong. Pires et al. also likely experimental). Yes (Liu et al. review of longitudinal studies).
10. Heavy daily use of smartphones and social media can cause attention fragmentation. ~2-3% ~10-12% Potentially (van Wezel et al. intervention, though manipulation failed, was field-based. “Mere presence” studies like Ward et al. are lab). Yes (Longitudinal studies like Ra et al., Deng et al. cited for associations with ADHD symptoms).
11. Attention fragmentation can cause a decline in mental health… ~2-3% ~5-7% Potentially (Intervention studies on disabling notifications like Fitz et al., Olson et al. are field-based and show some MH impact). Yes (Correlational studies on multitasking and MH like Becker et al., Shin et al.).
12. Heavy daily use of smartphones and social media can cause behavioral addiction. ~7-10% ~20-25% Yes (Allcott et al. 2022 is explicitly cited as a field study suggesting habit formation and self-control problems). Yes (Truzoli et al. on distress post-internet; Anderson & Wood on blunted reward sensitivity).
13. Behavioral addiction can cause a decline in mental health. ~5-7% ~50-55% No direct experimental field study cited for causing decline. The argument is often definitional or based on comorbidity. Yes (Hartmann & Blaszczynski longitudinal study on gambling; Petry et al., Ko et al. on comorbidity).
14. Heavy daily use of smartphones and social media can cause social deprivation… ~2-4% ~10-15% Yes (Allcott et al. 2020 deactivating Facebook was an experimental field study. Geraci et al. 2022 quasi-experimental study on internet speed is a natural experiment). Yes (Meta-analyses by Appel et al., Liu & Baumeister cited, though Cheng et al. found no overall association).
15. Chronic social deprivation can cause a decline in mental health. ~10-12% ~60-65% Yes (Killgore et al. 2020 on lockdown effects is a natural experiment. Intervention studies like Alvarez et al., Costello et al., Lai et al. on increasing social interaction improve MH and are likely field-based). Yes (Cacioppo et al. longitudinal study on isolation and depression).
17. Social media increases visual social comparisons among adolescent girls. ~2-3% ~15-20% No (Cited experimental studies like Fardouly et al. 2015, Kleemans et al. 2018, Tiggemann & Anderberg 2020 are lab/online, not field studies measuring social comparison as an outcome in a consequential real-world setting). Yes (Couture Bue 2020 eye-tracking study; Fardouly et al. 2017 survey).
18. Social media increases perfectionism among adolescent girls. ~2-3% ~5-8% No (Consensus states no experimental studies directly manipulating social media for this. Vanhoffelen et al. 2025 longitudinal study showed weak evidence). Yes (Surveys by Danielsen et al. 2024, Simon et al. 2022, Harren et al. 2021).
19. Social media increases relational aggression among adolescent girls… ~2-3% ~5-7% No (Consensus states no experimental evidence for SM use -> increased RA. Martins & Weaver meta-analysis included experiments on “exposure to RA -> RA behavior,” not direct SM manipulation). Yes (Marinoni et al. 2023 on cyberbullying vulnerability).
20. Among adolescent girls, social media increases exposure to other people displaying or discussing their mental disorders. ~3-5% ~15-20% No (Consensus states no direct causal link references. Cited studies are observational, e.g., phenomena on TikTok, Twitter disclosures). Yes (Alho et al. on peer networks; Hamilton et al. on self-harm content exposure).
21. Social media increases sexual predation and harassment of adolescent girls… ~1-2% ~10-15% Potentially (Bhuller et al. 2013 quasi-experimental study on general internet access and sex crimes is a natural experiment, but not specific to SM mechanisms for predation). Consensus states no SM-specific experimental studies. Yes (Bratton et al. 2020 on Facebook use and online harassment).
24. If most parents waited… it would benefit the mental health of adolescents overall. ~2-3% ~5-8% No (Cited studies are correlational/longitudinal. No experimental field studies manipulating delayed access). Yes (Dempsey et al., Vaterlaus et al., Gerosa et al., Sapien Labs, Sun 2023, all observational).
25. Imposing (and enforcing) a legal minimum age of 16… would benefit the mental health of adolescents overall. ~1-2% ~5-7% No (Consensus states no strictly related references. Cited experimental work like Ferguson 2024 is on temporary abstention, not age limits). Yes (General research linking SM to decreased MH like Twenge & Campbell 2018 cited as rationale, but it’s not testing the policy).
26. Phone-free schools would benefit the mental health of adolescents overall. ~5-7% ~10-15% Yes (Studies of school phone bans cited in Böttger & Zierer 2024 meta-analysis, King et al. 2024, Campbell et al. 2024 review, Goodyear et al. 2025 meta-analysis are natural experiments in field settings). No (The studies cited are evaluations of bans, not general observational studies of phone use in schools).

Notes on Estimation from Figure 3: The percentages for “% Causal Field” and “% Causal Field or Higher” are visual estimations from the provided stacked bar chart (Figure 3, p. 11 of the PDF). They are approximate due to the nature of reading from a graph.

My requests

I made a few requests and then had Gemini summarise. I put the final summary up top, the rest is in conversational order.

For each of the claims, there is a summary of the evidence in Supplementary Section 5. Look only at claims that make a causal claim. Then, note, for each claim, how many experts assessed the evidence as being causal, causal/field, replicated evidence, widespread causal evidence. Then, assess whether the studies cited in that section are causally informative, field studies or direct replications of one another. Note if there is a conflict with the expert assessment. Start with the claim that “Social media increases visual social comparison in girls”. We’ll take the claims in turn.

You’re right, Figure S3 (which is Figure 3 in the main text on page 11, not Supplementary Section S4 as I previously misremembered) is a stacked bar chart where the segments do appear to represent mutually exclusive portions of the respondents who answered the “level” question for each claim, likely categorized by the highest level of evidence they endorsed.

My apologies, my previous interpretation was incorrect based on how these charts are typically constructed to represent a whole. The explicit mention of allowing multiple answers is for the data collection phase, but for visualization in a stacked bar chart aiming for 100%, a recoding to the highest level endorsed is standard.

Let’s re-examine based on this understanding and your new request to specifically look for experimental field studies, natural experiments in the field, or well-controlled designs cited as supporting the causal claim in Section S5, and to report the “Causal Field” percentage directly from Figure 3.

Revised Interpretation of Figure 3 (formerly S3): The bars in Figure 3 sum to 100% for each claim (representing all experts who provided a “level of evidence” response for that claim). Each colored segment represents the proportion of those experts for whom that specific level was their highest endorsed level of evidence.

I will now go through the claims again, try to estimate the “Causal field” percentage from the visual in Figure 3 (page 11), and assess the S5 text for the types of evidence you’ve specified.

Revised Table

Claim % Causal Field (Est. from Fig 3) % Causal Field or Higher (Est. from Fig 3) S5 Text: Experimental Field, Natural Experiment, or Well-Controlled Design in Field Supporting Causal Claim? S5 Text: Observational Study Cited as Supporting Causal Claim?
8. Heavy daily use of smartphones and social media can cause sleep deprivation. ~10-12% ~25-30% Yes (Intervention studies like Martin et al., Perrault et al., Tu et al., described as improving sleep by reducing screen time, could be field experiments). Schechter et al. (meta-analysis of blue light interventions) and Hartstein et al. (consensus panel review) also point to experimental/interventional evidence. Yes (Longitudinal studies like Rod et al., Liu et al. cited for associations).
9. Chronic sleep deprivation can cause a decline in mental health. ~5-7% ~75-80% Yes (Scott et al. meta-analysis of sleep-improvement interventions showing mental health benefits is strong. Pires et al. also likely experimental). Yes (Liu et al. review of longitudinal studies).
10. Heavy daily use of smartphones and social media can cause attention fragmentation. ~2-3% ~10-12% Potentially (van Wezel et al. intervention, though manipulation failed, was field-based. “Mere presence” studies like Ward et al. are lab). Yes (Longitudinal studies like Ra et al., Deng et al. cited for associations with ADHD symptoms).
11. Attention fragmentation can cause a decline in mental health… ~2-3% ~5-7% Potentially (Intervention studies on disabling notifications like Fitz et al., Olson et al. are field-based and show some MH impact). Yes (Correlational studies on multitasking and MH like Becker et al., Shin et al.).
12. Heavy daily use of smartphones and social media can cause behavioral addiction. ~7-10% ~20-25% Yes (Allcott et al. 2022 is explicitly cited as a field study suggesting habit formation and self-control problems). Yes (Truzoli et al. on distress post-internet; Anderson & Wood on blunted reward sensitivity).
13. Behavioral addiction can cause a decline in mental health. ~5-7% ~50-55% No direct experimental field study cited for causing decline. The argument is often definitional or based on comorbidity. Yes (Hartmann & Blaszczynski longitudinal study on gambling; Petry et al., Ko et al. on comorbidity).
14. Heavy daily use of smartphones and social media can cause social deprivation… ~2-4% ~10-15% Yes (Allcott et al. 2020 deactivating Facebook was an experimental field study. Geraci et al. 2022 quasi-experimental study on internet speed is a natural experiment). Yes (Meta-analyses by Appel et al., Liu & Baumeister cited, though Cheng et al. found no overall association).
15. Chronic social deprivation can cause a decline in mental health. ~10-12% ~60-65% Yes (Killgore et al. 2020 on lockdown effects is a natural experiment. Intervention studies like Alvarez et al., Costello et al., Lai et al. on increasing social interaction improve MH and are likely field-based). Yes (Cacioppo et al. longitudinal study on isolation and depression).
17. Social media increases visual social comparisons among adolescent girls. ~2-3% ~15-20% No (Cited experimental studies like Fardouly et al. 2015, Kleemans et al. 2018, Tiggemann & Anderberg 2020 are lab/online, not field studies measuring social comparison as an outcome in a consequential real-world setting). Yes (Couture Bue 2020 eye-tracking study; Fardouly et al. 2017 survey).
18. Social media increases perfectionism among adolescent girls. ~2-3% ~5-8% No (Consensus states no experimental studies directly manipulating social media for this. Vanhoffelen et al. 2025 longitudinal study showed weak evidence). Yes (Surveys by Danielsen et al. 2024, Simon et al. 2022, Harren et al. 2021).
19. Social media increases relational aggression among adolescent girls… ~2-3% ~5-7% No (Consensus states no experimental evidence for SM use -> increased RA. Martins & Weaver meta-analysis included experiments on “exposure to RA -> RA behavior,” not direct SM manipulation). Yes (Marinoni et al. 2023 on cyberbullying vulnerability).
20. Among adolescent girls, social media increases exposure to other people displaying or discussing their mental disorders. ~3-5% ~15-20% No (Consensus states no direct causal link references. Cited studies are observational, e.g., phenomena on TikTok, Twitter disclosures). Yes (Alho et al. on peer networks; Hamilton et al. on self-harm content exposure).
21. Social media increases sexual predation and harassment of adolescent girls… ~1-2% ~10-15% Potentially (Bhuller et al. 2013 quasi-experimental study on general internet access and sex crimes is a natural experiment, but not specific to SM mechanisms for predation). Consensus states no SM-specific experimental studies. Yes (Bratton et al. 2020 on Facebook use and online harassment).
24. If most parents waited… it would benefit the mental health of adolescents overall. ~2-3% ~5-8% No (Cited studies are correlational/longitudinal. No experimental field studies manipulating delayed access). Yes (Dempsey et al., Vaterlaus et al., Gerosa et al., Sapien Labs, Sun 2023, all observational).
25. Imposing (and enforcing) a legal minimum age of 16… would benefit the mental health of adolescents overall. ~1-2% ~5-7% No (Consensus states no strictly related references. Cited experimental work like Ferguson 2024 is on temporary abstention, not age limits). Yes (General research linking SM to decreased MH like Twenge & Campbell 2018 cited as rationale, but it’s not testing the policy).
26. Phone-free schools would benefit the mental health of adolescents overall. ~5-7% ~10-15% Yes (Studies of school phone bans cited in Böttger & Zierer 2024 meta-analysis, King et al. 2024, Campbell et al. 2024 review, Goodyear et al. 2025 meta-analysis are natural experiments in field settings). No (The studies cited are evaluations of bans, not general observational studies of phone use in schools).

Notes on Estimation from Figure 3: The percentages for “% Causal Field” and “% Causal Field or Higher” are visual estimations from the provided stacked bar chart (Figure 3, p. 11 of the PDF). They are approximate due to the nature of reading from a graph.

Key Revisions and Observations:

Response Okay, this is a detailed task. I will go through each of the identified causal claims, extract the expert assessment of evidence levels, and then analyze the studies cited in the consensus statement for that claim to assess their nature (causally informative, field, replication) and identify any conflicts.

The definition of “Field evidence” provided in the document (page 24) is: “Evidence collected in real-world, consequential settings using objective measures (e.g., official statistics in real-world settings, such as suicide rates; self-reported surveys are not field evidence).” This is a strict definition. “Causally informative” will generally refer to experimental studies (lab, field, or well-designed interventions).

Let’s start with Claim 17 as requested, then proceed through the other causal claims.

Causal Claims from Table 1: * General Causal Claims: 8, 9, 10, 11, 12, 13, 14, 15 * Gender-related Causal Claims: 17, 18, 19, 20, 21 * Policy Recommendation Causal Claims: 24, 25, 26

Claim 17. Social media increases visual social comparisons among adolescent girls. (Consensus statement: p. 98-100)

Now, for the other causal claims:

Claim 8. Heavy daily use of smartphones and social media can cause sleep deprivation. (Consensus statement: p. 77-79)

Claim 9. Chronic sleep deprivation can cause a decline in mental health. (Consensus statement: p. 81)

Claim 10. Heavy daily use of smartphones and social media can cause attention fragmentation. (Consensus statement: p. 82-83)

Claim 11. Attention fragmentation can cause a decline in mental health (possibly through mediating factors such as its negative impact on social relationships). (Consensus statement: p. 84-85)

Claim 12. Heavy daily use of smartphones and social media can cause behavioral addiction. (Consensus statement: p. 87-88)

Claim 13. Behavioral addiction can cause a decline in mental health. (Consensus statement: p. 90)

Claim 14. Heavy daily use of smartphones and social media can cause social deprivation, such as isolation and lack of formative social experiences. (Consensus statement: p. 92-93)

Claim 15. Chronic social deprivation can cause a decline in mental health. (Consensus statement: p. 95)

Claim 18. Social media increases perfectionism among adolescent girls. (Consensus statement: p. 101-102) * Expert Assessment of Evidence Level (N=72): * No evidence: 23.61% * Correlational: 63.89% * Causal limited: 27.78% * Causal field: 9.72% * Replicated causal field: 5.55% * Wide-scale causal: 5.55%

Claim 19. Social media increases relational aggression among adolescent girls, for example by providing tools for cyberbullying and exclusion. (Consensus statement: p. 103-104)

Claim 20. Among adolescent girls, social media increases exposure to other people displaying or discussing their mental disorders. (Consensus statement: p. 105-106)

Claim 21. Social media increases sexual predation and harassment of adolescent girls, for example by providing predators with access to potential victims. (Consensus statement: p. 107-108)

Claim 24. If most parents waited until their children were in high school to give them their first smartphones, it would benefit the mental health of adolescents overall. (Consensus statement: p. 112-113)

Claim 25. Imposing (and enforcing) a legal minimum age of 16 for opening social media accounts would benefit the mental health of adolescents overall. (Consensus statement: p. 114-115)

Claim 26. Phone-free schools would benefit the mental health of adolescents overall. (Consensus statement: p. 116-117)

You are absolutely correct. Claims like “Social media increases…” are indeed causal claims, even if they aren’t explicitly about causing a decline in mental health. I will re-evaluate with that in mind, focusing on the “Causal field” expert assessment for all identified causal claims and explicitly noting if any cited studies are experimental field studies.

Let’s go through them again.

Claim 8. Heavy daily use of smartphones and social media can cause sleep deprivation. (Consensus statement: p. 77-79)

Claim 9. Chronic sleep deprivation can cause a decline in mental health. (Consensus statement: p. 81)

Claim 10. Heavy daily use of smartphones and social media can cause attention fragmentation. (Consensus statement: p. 82-83)

Claim 11. Attention fragmentation can cause a decline in mental health… (Consensus statement: p. 84-85)

Claim 12. Heavy daily use of smartphones and social media can cause behavioral addiction. (Consensus statement: p. 87-88)

Claim 13. Behavioral addiction can cause a decline in mental health. (Consensus statement: p. 90)

Claim 14. Heavy daily use of smartphones and social media can cause social deprivation… (Consensus statement: p. 92-93)

Claim 15. Chronic social deprivation can cause a decline in mental health. (Consensus statement: p. 95)

Claim 17. Social media increases visual social comparisons among adolescent girls. (Consensus statement: p. 98-100)

Claim 18. Social media increases perfectionism among adolescent girls. (Consensus statement: p. 101-102)

Claim 19. Social media increases relational aggression among adolescent girls… (Consensus statement: p. 103-104)

Claim 20. Among adolescent girls, social media increases exposure to other people displaying or discussing their mental disorders. (Consensus statement: p. 105-106)

Claim 21. Social media increases sexual predation and harassment of adolescent girls… (Consensus statement: p. 107-108)

Claim 24. If most parents waited… it would benefit the mental health of adolescents overall. (Consensus statement: p. 112-113)

Claim 25. Imposing (and enforcing) a legal minimum age of 16 for opening social media accounts would benefit the mental health of adolescents overall. (Consensus statement: p. 114-115)

Claim 26. Phone-free schools would benefit the mental health of adolescents overall. (Consensus statement: p. 116-117)

Summary of Experimental Field Studies & Expert Assessment:

It seems for many claims directly testing “Social media increases X” with experimental field studies, the evidence is sparse or non-existent according to the cited literature. The expert ratings for “Causal Field” are generally low for these, but sometimes higher than what the cited experimental field literature in the consensus statements would directly support, suggesting experts might be drawing on a wider body of knowledge or interpreting observational field data more causally. For policy-related claims, the expert assessment of causal field evidence is also very low, reflecting the difficulty and rarity of such large-scale experimental policy evaluations.

Request for summary table First draft:

So, experts could endorse multiple levels of evidence (despite some of them being mutually exclusive). Do the authors describe how they recoded these responses to make a graph where categories sum to 100%? Highest/lowest level of evidence endorsed?

Second draft:

Earlier I asked for experimental field studies, but causal evidence could also come from natural experiments or well-controlled designs. Please revise the table and assess whether an experiment or natural experiment in the field supporting the claim is reported. If the text in Section S5 makes the case that an observational study can be used to back up the causal claim, mention it. Also, don’t sum up the reported percentages for causal field and above since they are not mutually exclusive. Instead, try to read it off the plot and also report the percentage who say causal field in another column.

  1. In reality, this claim in the graph pertains only to the US, surely they could make room for that important qualification.↩︎

  2. Jay van Bavel later deleted this tweet and said he was not trying to equate their consensus to the consensus around climate change. I would say he tried… and had to walk it back.↩︎

  3. I manually vetted only one claim (about body image/visual social comparison), so Gemini could get this wrong (e.g. misclassify a study), but I think distilling documents is one of the things it does rather well.↩︎

Corrections

If you see mistakes or want to suggest changes, please create an issue on the source repository.

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY 4.0. Source code is available at https://github.com/rubenarslan/rubenarslan.github.io, unless otherwise noted. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".

Citation

For attribution, please cite this work as

Arslan (2025, May 20). One lives only to make blunders: Consensus: How the sausage gets made. Retrieved from https://rubenarslan.github.io/posts/2025-05-20-consensus-how-the-sausage-gets-made/

BibTeX citation

@misc{arslan2025consensus:,
  author = {Arslan, Ruben C.},
  title = {One lives only to make blunders: Consensus: How the sausage gets made},
  url = {https://rubenarslan.github.io/posts/2025-05-20-consensus-how-the-sausage-gets-made/},
  year = {2025}
}

© Copyright 2018-2023 Ruben Arslan. Privacy Policy.