Toolbox: Conceptual overview
Last updated: 2024-07-01
Checks: 7 0
Knit directory: toolbox/
This reproducible R Markdown analysis was created with workflowr (version 1.7.1). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.
Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.
Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.
The command set.seed(20220228) was run prior to running
the code in the R Markdown file. Setting a seed ensures that any results
that rely on randomness, e.g. subsampling or permutations, are
reproducible.
Great job! Recording the operating system, R version, and package versions is critical for reproducibility.
Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.
Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.
Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.
The results in this page were generated with repository version 6537266. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.
Note that you need to be careful to ensure that all relevant files for
the analysis have been committed to Git prior to generating the results
(you can use wflow_publish or
wflow_git_commit). workflowr only checks the R Markdown
file, but you know if there are other scripts or data files that it
depends on. Below is the status of the Git repository when the results
were generated:
Ignored files:
Ignored: .Rproj.user/
Unstaged changes:
Modified: data/toolbox_evidence.xlsx
Modified: output/figure_map.pdf
Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.
These are the previous versions of the repository in which changes were
made to the R Markdown (analysis/table_concept.Rmd) and
HTML (public/table_concept.html) files. If you’ve
configured a remote Git repository (see ?wflow_git_remote),
click on the hyperlinks in the table below to view the files as they
were in that past version.
| File | Version | Author | Date | Message |
|---|---|---|---|---|
| html | 65dbb77 | Kozyreva | 2024-05-21 | update link in debunking |
| html | c5033f2 | Kozyreva | 2024-05-13 | Build site. |
| html | 466dd67 | Kozyreva | 2024-05-13 | publication update |
| html | e7e6f57 | Kozyreva | 2024-04-24 | Build site. |
| html | 8892615 | Kozyreva | 2024-04-24 | small update to the title and evidence table |
| html | 5bae2a2 | Kozyreva | 2024-02-20 | Build site. |
| html | c384b21 | Kozyreva | 2024-02-15 | minor changes to concept table |
| html | 1a1a155 | Kozyreva | 2023-11-30 | minor updates |
| Rmd | 5be9264 | Kozyreva | 2023-11-24 | what should be the final version of the toolbox 2.0 |
| html | 5be9264 | Kozyreva | 2023-11-24 | what should be the final version of the toolbox 2.0 |
| html | 22a40df | Kozyreva | 2023-10-24 | v2 updates |
| html | 663ae9c | Kozyreva | 2023-10-24 | Build site. |
| Rmd | 3efec23 | Kozyreva | 2023-10-24 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| Rmd | c2b8b90 | Kozyreva | 2023-10-12 | v2 branch update |
| html | c2b8b90 | Kozyreva | 2023-10-12 | v2 branch update |
| html | 168d4c7 | Kozyreva | 2023-02-28 | Build site. |
| html | 8c38442 | Anastasia Kozyreva | 2023-01-05 | Build site. |
| html | 0b02534 | Anastasia Kozyreva | 2022-12-16 | Build site. |
| Rmd | e57774a | Anastasia Kozyreva | 2022-12-05 | update |
| html | 295f580 | Anastasia Kozyreva | 2022-12-05 | Build site. |
| Rmd | 1a9c195 | Anastasia Kozyreva | 2022-12-01 | update and folder reorg |
| html | 2b13b62 | Anastasia Kozyreva | 2022-12-01 | Build site. |
| html | b8df76e | Anastasia Kozyreva | 2022-11-16 | Build site. |
| html | 51ffb87 | Anastasia Kozyreva | 2022-11-16 | Build site. |
| Rmd | 1fcecbd | Anastasia Kozyreva | 2022-11-16 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 421eeda | Anastasia Kozyreva | 2022-11-08 | Build site. |
| html | 355e92a | Anastasia Kozyreva | 2022-10-11 | Build site. |
| html | e3720bd | Anastasia Kozyreva | 2022-10-11 | tables update |
| html | 9986d6d | Anastasia Kozyreva | 2022-08-22 | update |
| Rmd | 0cec29f | Anastasia Kozyreva | 2022-08-17 | tables update |
| html | 0cec29f | Anastasia Kozyreva | 2022-08-17 | tables update |
| html | 22dac54 | Anastasia Kozyreva | 2022-08-04 | no message |
| html | b794099 | Anastasia Kozyreva | 2022-08-04 | tables update |
| Rmd | a58ab0a | Anastasia Kozyreva | 2022-07-28 | Minor updates |
| html | a58ab0a | Anastasia Kozyreva | 2022-07-28 | Minor updates |
| Rmd | 7c28e67 | Anastasia Kozyreva | 2022-07-28 | design and tables update |
| html | 7c28e67 | Anastasia Kozyreva | 2022-07-28 | design and tables update |
| html | 4effdc5 | Anastasia Kozyreva | 2022-07-27 | Build site. |
| Rmd | d069c9f | Anastasia Kozyreva | 2022-07-27 | update |
| html | d069c9f | Anastasia Kozyreva | 2022-07-27 | update |
| html | 8c3b8f3 | Anastasia Kozyreva | 2022-07-25 | json code update |
| Rmd | d8a3618 | Anastasia Kozyreva | 2022-07-21 | update |
| html | d8a3618 | Anastasia Kozyreva | 2022-07-21 | update |
| html | f182d72 | Anastasia Kozyreva | 2022-07-20 | Build site. |
| Rmd | de4399c | Anastasia Kozyreva | 2022-07-20 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | f9405d6 | Anastasia Kozyreva | 2022-06-27 | Build site. |
| html | fe780ff | Anastasia Kozyreva | 2022-06-27 | Build site. |
| html | 8ab172c | Anastasia Kozyreva | 2022-06-08 | Build site. |
| html | 2eaf831 | Anastasia Kozyreva | 2022-06-02 | Build site. |
| Rmd | 4f9aedf | Anastasia Kozyreva | 2022-06-02 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | adf0386 | Anastasia Kozyreva | 2022-05-20 | Build site. |
| Rmd | d321524 | Anastasia Kozyreva | 2022-05-20 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 33c2f64 | Anastasia Kozyreva | 2022-05-19 | Build site. |
| Rmd | ad71414 | Anastasia Kozyreva | 2022-05-19 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 77a734a | Anastasia Kozyreva | 2022-05-17 | Build site. |
| Rmd | dad74e3 | Anastasia Kozyreva | 2022-05-17 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 1504401 | Anastasia Kozyreva | 2022-05-16 | Build site. |
| Rmd | 697b957 | Anastasia Kozyreva | 2022-05-16 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 7a81fdc | Anastasia Kozyreva | 2022-04-28 | Build site. |
| Rmd | 3f7fe5d | Anastasia Kozyreva | 2022-04-28 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | c79c858 | Anastasia Kozyreva | 2022-04-27 | Build site. |
| Rmd | b8f7c8d | Anastasia Kozyreva | 2022-04-27 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 280c6e4 | Anastasia Kozyreva | 2022-04-27 | Build site. |
| Rmd | 3524d88 | Anastasia Kozyreva | 2022-04-27 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | c35a82d | Anastasia Kozyreva | 2022-04-27 | Build site. |
| Rmd | 47e86c9 | Anastasia Kozyreva | 2022-04-27 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 14c638f | Anastasia Kozyreva | 2022-03-01 | Build site. |
| Rmd | 658b5b4 | Anastasia Kozyreva | 2022-03-01 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
| html | 7ea496e | Anastasia Kozyreva | 2022-02-28 | Build site. |
| html | 17e4292 | Anastasia Kozyreva | 2022-02-28 | Build site. |
| html | 6ab970b | Anastasia Kozyreva | 2022-02-28 | Build site. |
| html | f157d6c | Anastasia Kozyreva | 2022-02-28 | Build site. |
| Rmd | 94e9bae | Anastasia Kozyreva | 2022-02-28 | wflow_publish(files = files, republish = TRUE, delete_cache = TRUE, |
This part of the online supplement is a digital toolbox in the form of a dynamic table that offers a conceptual overview of interventions. For useful links and other details click on the arrow or the “Expand” button to the left of the intervention type. To search through the whole table, use the search function below. Use the smaller search field under a column’s header to search within that column. You can sort a column by clicking on its header, or sort multiple columns by holding the shift key while sorting.
Details
Intervention
Description
Example
Conceptual framework
Category
Problem
Targeted outcomes
Audience
Implementation
Scalability
Limitations
Accuracy prompts are used to shift people's attention to the concept of accuracy.
Asking people to evaluate the accuracy of a headline or showing people a video about the importance of sharing only accurate content.
Limited-attention utility model (Pennycook et al., 2021)
Nudge
Sharing false and misleading content
Behavior: thinking about accuracy before sharing information online
Social media users
Easy to implement. Can be implemented as a social media prompt (e.g., a pop-up asking the person to evaluate the accuracy of a headline or a short reminder about importance of accuracy before sharing).
3 - High scalability in principle; scalability improves with cooperation with platforms.
Short-term intervention.
Does not improve people’s ability to discern misinformation.
Requires some underlying capacity to recognize whether information is accurate.
Debunking and rebuttals are strategies aimed at dispelling misconceptions and countering false beliefs. Debunking involves offering corrective information to address a specific misconception. Rebuttals, particularly in the context of science denialism, consist of presenting accurate facts related to a topic that has been inaccurately addressed (topic rebuttal) or exposing the rhetorical tactics often employed to reject established scientific findings (technique rebuttal).
Debunking can be implemented in four steps: (1) state the truth, (2) warn about imminent misinformation exposure, (3) specify the misinformation and explain why it is wrong, (4) reinforce the truth by offering the correct explanation. Depending on circumstances (e.g., availability of a pithy fact), starting with step 2 may be appropriate.
Persuasion theory; models of belief change (Ecker et al., 2022)
Refutation strategy
Misinformation that can be fact-checked
Belief calibration; Competence: Detecting and resisting manipulative and false information
General public
Implementation is flexible but effortful. Requires professional fact-checkers or a trusted source. Can also be implemented in a collective and transparent enterprise (e.g., Wikipedia). Automation is possible.
Has been implemented by social media platforms including Twitter, Facebook, TikTok.
1 - Limited scalability.
Reactive. Topic-specific. Belief regression. Limited scalability. Recipient attention required.
Friction makes relevant processes slower or more effortful by design.
Asking people to pause and think before sharing content on social media. This could be as simple as a short prompt, e.g., "Want to read this before sharing?"
Data friction (Bates, 2017)
Nudge
Sharing false and misleading content
Behavior: Pausing rather than acting on initial impulse
Social media users
Easy to implement. Has been implemented by social media platforms including Twitter, Facebook, TikTok.
3 - High scalability.
Unclear how to legislate or regulate.
Potential for abuse: Similar techniques can also be used to restrict freedom of choice and communication on the internet (e.g., authoritarian regimes use friction to limit citizens’ access to information).
Limited access to evidence: Data on many of the relevant tests are proprietary and not publically available (e.g., Twitter, TikTok).
Inoculation is a preemptive intervention that exposes people to a weakened form of common misinformation and/or manipulation strategies in order to build up their ability to resist them.
Teaching people about the strategy of using ``fake experts" (presenting unqualified people as credible) in order to increase their recognition of and resilience to this strategy.
Inoculation theory (McGuire, 1964)
Refutation strategy, Boost
Misinformation that makes use of manipulation strategies (e.g., conspiracy theories, false information)
Belief calibration; Competence: Detecting and resisting manipulative and false information
General public (age 8+), Social media users, Educators
Implementation ease varies from easy to effortful (depending on the format). Has been implemented in various forms, incl. warning messages, games, and videos, and used by the WHO and UN in anti-misinformation campaigns.
2 - Moderate scalability.
Scalability: Inoculation interventions generally require some level of buy-in (e.g., playing a game, watching a video, reading a text).
Gamified inoculation requires users to understand how the game works.
Effects robust for a decrease in false-claim belief, but not always for real news; so far relying on self-reported intentions.
Verification strategies for evaluating online information encompass a range of techniques and methods used to assess the credibility, accuracy, and reliability of digital content. Lateral reading is a strategy that professional fact checkers use to evaluate online sources: Investigate the credibility of a website by searching for information about it on the internet. Additional verification strategies can include image searching and tracing the original context of the information.
School-based interventions with instructional strategies like teacher modeling and guided practice can be used to teach lateral reading. Pop-up graphics can also be used to prompt social media users to read laterally.
Civic Online Reasoning (Wineburg et al., 2022)
Educational intervention, Boost
False and misleading content (especially from unfamiliar sources)
Competence: Evaluating the credibility of online sources.
High school and university students; General public
Education: Implementation is flexible but effortful. Has been implemented in school curricula and short educational videos. Online (e.g., social media): Easy to implement. Could be presented in various formats (pop-up, banner, window, etc.) that appear on top of content in response to user interaction (e.g., sharing or commenting).
Education: 1 - Limited scalability; can be integrated in existing courses or taught as a separate unit. Online: 3 - High scalability; similar approaches (e.g., tagging) have already been implemented by various social media platforms.
Education: Takes time to teach (if implemented in school curricula).
Online: Requires people to be engaged and motivated to learn and to implement the strategy.
Media-literacy tips give people a list of strategies for identifying false and misleading information in their newsfeeds.
Facebook offers tips to spot false news, including "be sceptical of headlines," "look closely at the URL," and "investigate the source."
Not identified
Boost
False and misleading content
Competence: Media literacy and social media skills
General public (refinements should consider subgroups, e.g., students, senior citizens)
Can be easy to implement but also potentially resource-intensive (e.g., in face-to-face settings). Has been implemented by social media platforms (e.g., Facebook). Could be implemented online in a pop-up via plug-ins or other tools, as part of media campaigns, and in person.
3 - High scalability for basic tips; 2 - Moderate scalability for more educational versions, which are resource-intensive.
Decay.
Modest effect sizes.
Social norms leverage social information (peer influence) to encourage people not to believe, endorse, or share misinformation.
Emphasizing that most people of a given group disapprove of sharing or using false information (descriptive norm) and/or that such actions are generally considered wrong, inappropriate, or harmful (inductive norm).
Social norms theory/theory of normative conduct (Cialdini, 1991)
Nudge
Believing and especially sharing false and misleading content
Behavior: Following normative beliefs, e.g., when sharing information online
General public (specifically, people who share or believe in misinformation)
Easy to implement if relevant/reliable social norms are available. Could easily be implemented on social media (e.g., placing intervention text above links to articles in social media feeds).
3 - High scalability.
Effect may be moderated by social distance, identification with the group, or lack/suitability of an existing norm.
Source-credibility labels show how a particular news source was rated by professional fact-checking organizations.
NewsGuard labels indicate the trustworthiness of news and information websites with a reliability rating from 0 to 100, based on 9 journalistic criteria that assess basic practices of reliability and transparency.
Persuasion theory; models of belief change
Refutation strategy; Nudge
False and misleading content (especially from unfamiliar sources)
Belief calibration; Detecting untrustworthy sources of information
General public
Easy to implement. Can be installed as a browser add-on.
3 - High scalability.
Limited efficacy for users with healthy information diets.
Warning labels explicitly alert individuals to the possibility of being misled by a particular piece of information or its source. Fact-checking labels indicate the trustworthiness rating assigned to a piece of content by professional fact-checkers.
Facebook adds labels "False (Independent fact-checkers say this information has no basis in fact)" or "Partly false (Independent fact-checkers say this information has some factual inaccuracies)".
Persuasion theory; models of belief change
Refutation strategy; Nudge
False and misleading content
Belief calibration; Detecting false or other types of problematic information
General public
Easy to implement. Has been implemented by various social media platforms (e.g., Twitter, Facebook) where information is reposted from external sources, as well as on news outlets.
3 - High scalability; already implemented on a large scale (as a mix of automated content detection and collaboration with fact-checking organizations).
Difficult to match the speed at which misinformation can be fabricated and spread.
Can produce "implied truth effect," in which warnings on a subset of fake news stories may increase the perceived accuracy of stories without warnings.
Warnings do not eliminate the continued influence of misinformation.
Privacy policy - Imprint/Provider Identification
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8
[2] LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
time zone: Europe/Berlin
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] reactablefmtr_2.0.0 htmltools_0.5.6 glue_1.6.2
[4] png_0.1-8 reactable_0.4.4 DT_0.29
[7] formattable_0.2.1 shiny_1.7.5 questionr_0.7.8
[10] ggthemes_4.2.4 webshot_0.5.5 wesanderson_0.3.6
[13] janitor_2.2.0 sjlabelled_1.2.0 knitr_1.43
[16] kableExtra_1.3.4 broom_1.0.5 jtools_2.2.2
[19] readxl_1.4.3 here_1.0.1 lubridate_1.9.2
[22] forcats_1.0.0 stringr_1.5.0 dplyr_1.1.3
[25] purrr_1.0.2 readr_2.1.4 tidyr_1.3.0
[28] tibble_3.2.1 ggplot2_3.5.1 tidyverse_2.0.0
[31] pacman_0.5.1 workflowr_1.7.1
loaded via a namespace (and not attached):
[1] tidyselect_1.2.0 viridisLite_0.4.2 fastmap_1.1.1 promises_1.2.1
[5] labelled_2.12.0 digest_0.6.33 timechange_0.2.0 mime_0.12
[9] lifecycle_1.0.3 ellipsis_0.3.2 processx_3.8.2 magrittr_2.0.3
[13] compiler_4.3.1 rlang_1.1.1 sass_0.4.7 tools_4.3.1
[17] utf8_1.2.3 yaml_2.3.7 htmlwidgets_1.6.2 xml2_1.3.5
[21] miniUI_0.1.1.1 withr_2.5.0 grid_4.3.1 fansi_1.0.4
[25] git2r_0.32.0 xtable_1.8-4 colorspace_2.1-0 scales_1.3.0
[29] insight_0.20.1 cli_3.6.1 rmarkdown_2.24 crayon_1.5.2
[33] generics_0.1.3 rstudioapi_0.15.0 httr_1.4.7 tzdb_0.4.0
[37] cachem_1.0.8 pander_0.6.5 rvest_1.0.3 cellranger_1.1.0
[41] vctrs_0.6.3 jsonlite_1.8.7 callr_3.7.3 hms_1.1.3
[45] crosstalk_1.2.0 systemfonts_1.0.4 jquerylib_0.1.4 reactR_0.4.4
[49] ps_1.7.5 stringi_1.7.12 gtable_0.3.4 later_1.3.1
[53] munsell_0.5.0 pillar_1.9.0 R6_2.5.1 rprojroot_2.0.3
[57] evaluate_0.21 haven_2.5.3 highr_0.10 backports_1.4.1
[61] snakecase_0.11.1 httpuv_1.6.11 bslib_0.5.1 Rcpp_1.0.11
[65] svglite_2.1.1 whisker_0.4.1 xfun_0.40 fs_1.6.3
[69] getPass_0.2-2 pkgconfig_2.0.3