Documented releases TikTok toolkit

TLDR

• Documented published an open-source, four-step pipeline to sample, download, transcribe and cluster TikTok videos.¹
• Step one pools participants' TikTok viewing histories, downloaded as JSON, after consent and removal of personal data.¹
• Transcription uses open-source Whisper, but the default translation step uses Google Translate, and scraping can drop videos or audio.¹

Documented, a newsroom that spent six months analysing immigration-related content on TikTok, published a technical guide on 23 October releasing the code behind that work for others to reuse.¹

Why it matters: this is a platform-methodology item. Documented says TikTok places tight restrictions on systematic analysis of its content feed, and that reposting makes it hard to trace where a video came from. The pipeline works around both by building a sample from volunteers' own data and then clustering transcripts to link reuploads to originals.¹

Driving the news:

• Step 1: participants request their viewing history from TikTok, ticking Your Activity and choosing JSON. A Python script, run in Google Colab or on a desktop, strips personal identifying information and outputs video URLs with watch dates.¹
• Step 2: a scraper built on yt-dlp downloads videos from a CSV of links and records author, post date and play count. Documented says rate limiting, behavioural tracking and connectivity problems can leave videos or audio missing, so it advises running batches.¹
• Step 3: Whisper transcribes each video and the deep-translator library produces English text, using Google Translate by default. Documented says the models can run on a user's own computer.¹
• Step 4: DBSCAN clusters transcripts using containment similarity, which catches clipped re-uploads, and semantic similarity, with adjustable weights.¹

By the numbers:

• A sample of 10,000 TikTok posts over six months.¹
• 21 accounts using one lawyer's name, likeness or videos, 16 still active, per the companion piece.²

The big picture: Documented says the clustering exposed networks of copycat accounts ripping off real immigration lawyers, some scamming immigrants into sending money.¹ The companion investigation, published on 21 October, says TikTok deactivated one impersonator account around 15 September after Documented's press inquiry. It adds that Documented worked with the NYU Center for Social Media, AI, and Politics to access an exclusive collection of tens of millions of TikTok video URLs.² A TikTok spokesperson told Documented it does not allow impersonation, spam or scams.²

The other side: the guide stresses consent and anonymisation when handling other people's TikTok data, and says its own dataset was built on parameters Documented defined, so results depend on how a sample and the clustering weights are chosen.¹

Single-source note: both sources are Documented's own work, and no independent coverage or testing of the tools turned up. The extracted guide text had no code link, so the code was not reviewed here.¹²