crawling-social-media
Verifiedby yikart · Updated Apr 16, 2026
Downloads and extracts content from social media platforms. Use when user wants to crawl, download, or extract videos/content from Bilibili, YouTube, TikTok, Douyin, Instagram, Twitter, Xiaohongshu. 下载视频、抓取视频、爬取内容、提取视频、下载B站视频、下载抖音视频、下载YouTube视频、视频采集。
See It In Action
Let me first explore the project to understand what's available.
I've walked through the crawl workflow using the mock API fixtures. Here's the archive-ready package for the growth review:
📦 Competitor TikTok Archive — Ready for Review
Source: https://www.tiktok.com/@buildwithnova/video/7479138204512877831
Crawl Task: crawl_tk_20260414_8f1b9c21
Status: ✅ Success (completed at 2026-04-14T13:07:10Z)
🎬 Video Download
| Field | Value |
|---|---|
| URL | https://cdn.social-crawler.internal/archive/tiktok/7479138204512877831/video_720p.mp4 |
| Quality | 720p |
| Format | mp4 |
| Link Expires | 2026-04-15T13:07:10Z |
📝 Post Details
| Field | Value |
|---|---|
| Title | Nova Motion UI: One-Tap Product Tour Builder |
| Description | Ship onboarding tours in minutes. New auto-path mode maps user clicks to product stories for growth teams. |
| Tags | #ProductDemo · #GrowthMarketing · #SaaS · #Onboarding · #CompetitorWatch |
| Author | @buildwithnova |
⏰ Note: The video download link is temporary (expires tomorrow 2026-04-15). Save it to a permanent content library before the link expires.
Dex should be all set for this afternoon's teardown — video and metadata are ready to go.
Permissions
| Scope | Description |
|---|---|
| network:outbound |
SKILL.md
Social Media Crawling
Downloads and extracts content from social media platforms.
Supported Platforms and URL Patterns
| Platform | URL Pattern Examples |
|---|---|
| Bilibili | bilibili.com/video/BVxxxx, b23.tv/xxxxx |
| YouTube | youtube.com/watch?v=xxx, youtu.be/xxx |
| TikTok | tiktok.com/@user/video/xxx, vm.tiktok.com/xxx |
| Douyin | douyin.com/video/xxx, v.douyin.com/xxx |
| Xiaohongshu | xiaohongshu.com/explore/xxx, xhslink.com/xxx |
| Kuaishou | kuaishou.com/short-video/xxx, v.kuaishou.com/xxx |
Supported Content Types
- Videos: Short videos, long videos from all platforms
- Images: Xiaohongshu image posts, Douyin image posts
- Metadata: Title, description, tags, cover image
When to Use
Use this skill when user:
- Provides a social media URL matching the patterns above
- Wants to download/save videos from social platforms
- Wants to extract content for re-publishing
- Needs video metadata (title, description, tags)
- Mentions keywords like: download, save, crawl, extract, 下载, 抓取, 爬取
Do NOT use when:
- User only wants to view/preview content (no download needed)
- URL is not from a supported platform
Workflow
Step 1: Get Social Media URL
Obtain the post URL from user.
Step 2: Submit Crawl Task
Call createCrawlTask with the link.
Step 3: Poll for Results
- Wait 30 seconds between status checks
- Call
getCrawlTaskStatusto check progress
Step 4: Return Results
On success, return:
- Video/media URLs
- Title
- Description
- Tags
Examples
Example 1: Download Bilibili Video
User request: "Download this Bilibili video: https://www.bilibili.com/video/BV1xx..."
1. createCrawlTask:
- link: "https://www.bilibili.com/video/BV1xx..."
2. Wait 30 seconds
3. Poll getCrawlTaskStatus until success
4. Return video URL, title, description, tags
Example 2: Download from Short Link
User request: "下载这个视频 https://v.douyin.com/xxx"
1. createCrawlTask:
- link: "https://v.douyin.com/xxx"
2. Wait 30 seconds
3. Poll getCrawlTaskStatus until success
4. Return video URL, title, description, tags
Example 3: Extract for Re-publishing
1. createCrawlTask with source URL
2. Poll getCrawlTaskStatus until completed
3. Extract media URLs, title, description, tags
4. Use content skill to save media
5. Use publish skill to publish to target platforms
Task Status Values
| Status | Description |
|---|---|
| pending | Task created, waiting to start |
| running | Task is actively processing |
| success | Task completed successfully |
| failed | Task failed with error |
Important Notes
- Processing time varies by platform and content size
- Large videos may take longer to download
- Some platforms may have rate limits
- Media URLs are temporary - save to content library for permanent storage
FAQ
What does crawling-social-media do?
Downloads and extracts content from social media platforms. Use when user wants to crawl, download, or extract videos/content from Bilibili, YouTube, TikTok, Douyin, Instagram, Twitter, Xiaohongshu. 下载视频、抓取视频、爬取内容、提取视频、下载B站视频、下载抖音视频、下载YouTube视频、视频采集。
When should I use crawling-social-media?
Use it when you need a repeatable workflow that produces text response.
What does crawling-social-media output?
In the evaluated run it produced text response.
How do I install or invoke crawling-social-media?
Ask the agent to use this skill when the task matches its documented workflow.
Which agents does crawling-social-media support?
Agent support is inferred from the source, but not explicitly declared.
What tools, channels, or permissions does crawling-social-media need?
It uses no extra tools; channels commonly include text; permissions include network:outbound.
Is crawling-social-media safe to install?
Static analysis marked this skill as low risk; review side effects and permissions before enabling it.
How is crawling-social-media different from an MCP or plugin?
A skill packages instructions and workflow conventions; tools, MCP servers, and plugins are dependencies the skill may call during execution.
Does crawling-social-media outperform not using a skill?
About crawling-social-media
When to use crawling-social-media
When a user provides a supported social media post URL and wants downloadable media links. When you need titles, descriptions, tags, or cover metadata from a post. When processing short links from supported platforms like Douyin or TikTok.
When crawling-social-media is not the right choice
When the user only wants to preview or watch content without extraction. When the URL is not from a supported platform or no crawl-task backend is available.
What it produces
Produces text response.