{"data":{"id":2309,"lang":"en","title":"Researcher Scrapes 4.5 Billion TikTok Videos, Publishes Dataset on Hugging Face","summary":"An independent researcher scraped 4.5 billion TikTok video records over three weeks and published them as a public dataset on Hugging Face, despite acknowledging the pull broke TikTok's terms of service.","category":"technology","category_label":"Technology","published_at":"2026-09-04T02:01:17Z","updated_at":"2026-09-04T02:01:17Z","url":"https://techtools.cz/tools/news/article/2309-researcher-scrapes-4-5-billion-tiktok-videos-publishes-dataset-on-hugging-face","api_url":"https://techtools.cz/news-api/v1/articles/2309?lang=en","markdown_url":"https://techtools.cz/news-api/v1/articles/2309.md?lang=en","image_url":"https://images.unsplash.com/photo-1677442136019-21780ecad995?w=800","word_count":154,"reading_time_minutes":1,"sources":[{"url":"https://aiweekly.co/alerts/independent-researcher-uploads-45b-record-tiktok-video-dataset-to-hugging-face","domain":"aiweekly.co"},{"url":"https://huggingface.co/datasets/kuben-developer/tiktok-videos-4b","domain":"huggingface.co"},{"url":"https://news.ycombinator.com/item?id=49548625","domain":"news.ycombinator.com"}],"translation":{"lang":"cs","id":2382,"title":"Výzkumník stáhl 4,5 miliardy videí z TikToku a zveřejnil je jako dataset na Hugging Face","url":"https://techtools.cz/tools/news/clanek/2382-vyzkumnik-stahl-4-5-miliardy-videi-z-tiktoku-a-zverejnil-je-jako-dataset-na","api_url":"https://techtools.cz/news-api/v1/articles/2382?lang=cs"},"content":"An independent developer known as \"kuben-developer\" spent three weeks pulling data from TikTok's private Android API and published the results as a public dataset on Hugging Face: 4.5 billion video records packed into 27 compressed files totaling roughly 289 gigabytes.\n\nThe scrape used anonymous device registrations that required no login, reaching 27 of TikTok's 32 storage partitions. Each record includes video metadata, captions, tagged accounts, sound IDs, engagement counts (views, likes, comments, shares, saves), and geographic and language tags, but the researcher deliberately excluded usernames, profile data, and media URLs.\n\nThe dataset's own documentation acknowledges the collection violated TikTok's terms of service and warns that captions count as personal data under GDPR. The listing bans using the data to identify, profile, or target individuals and restricts it to research and educational use, though the engagement figures are one-time snapshots rather than tracked over time, and the country/language tags are TikTok's own unverified inferences."}}