Skip to main content
Ctrl+K

CATLISM | Online Compendium

Ctrl+K
  • On scripts and tools
  • Using the online compendium
  • From the book
    • Reference list for online contents
    • Content updates
    • Preservation copies of online materials
    • Errata
    • Copy-pasting code from the PDF version of the book
  • Setting up the working environment
    • Using conda
  • Metadata evaluation
    • Facebook metadata
      • Facebook metadata for posts
      • Facebook metadata for profiles
      • Facebook metadata for groups
    • Instagram metadata
      • Instagram metadata for posts
      • Instagram metadata for comments
    • Twitter metadata
      • Twitter metadata
    • Youtube metadata
      • Youtube metadata for videos
      • Youtube metadata for comments
  • Data collection
    • General purpose scrapers
      • #LancsBox
      • Archivebox
      • Trafilatura
      • BeautifulSoup
    • Social Media Platforms
      • Facebook
      • Instagram
      • Twitter
      • Youtube
  • Data processing
    • Date, time, and Unix
    • Text normalisation
    • PDF, Word, images
    • Language detection
    • Emoticons and emojis
    • Hashtags (word segmentation)
    • Other elements
    • Annotations
    • Verticalised (.vrt) format
  • Data exploration
    • OpenRefine
  • Data preservation
    • Wayback Machine
    • Git 101: the basics
  • Case-studies: CATLISM practical applications
    • Analysing crypto-drug market fora
    • Analysing the language of far-right groups on Twitter and Facebook
    • The communicative modus operandi of online child sexual groomers
  • Digital technicalities: a reading list
  • FAQs
  • Acknowledgments
  • Changelog
  • References
  • .md

Data collection

Data collection#

  • General purpose scrapers
    • #LancsBox
    • Archivebox
    • Trafilatura
    • BeautifulSoup
  • Social Media Platforms
    • Facebook
    • Instagram
    • Twitter
    • Youtube

previous

Youtube metadata for comments

next

General purpose scrapers

By Matteo Di Cristofaro

© Copyright 2023-2024, Matteo Di Cristofaro.

Built with Sphinx book theme
Privacy