mirror of
https://github.com/rohitg00/skillkit.git
synced 2026-11-18 18:04:26 +00:00
Cuts Vercel hobby edge-request burn on both skillkit (Vite static) and skillkit-docs (Next.js fumadocs) projects. Site stays human-first AND agent-first — Firecrawl, Context7, Crawl4AI, OpenClaw, Hermes, ChatGPT-User, Claude-User, PerplexityBot explicitly allowed; training crawlers and SEO scrapers blocked. skillkit (Vite, docs/skillkit): - vercel.json: Cache-Control on assets (1d browser, 7d edge, 30d SWR; immutable for hashed /assets/*), HTML (5min browser, 1d edge, 7d SWR), and /api JSON - redirects with has user-agent: deflect known training+SEO bots to /robots.txt; negative lookahead on source prevents /robots.txt redirect loop - public/robots.txt: explicit allow + deny lists skillkit-docs (Next.js, docs/fumadocs): - next.config.mjs: headers() block for /_next/static (1y immutable), assets (1d/7d/30d), /docs/* and / (5min/1d/7d) - src/middleware.ts: UA-aware allow/deny pipeline; allowed agents pass through, blocked bots get 403 with cache header so Vercel edge serves the rejection cheaply - public/robots.txt: same allow/deny list as marketing site