Files
Rohit Ghumare e42ae9d87a chore: edge cache headers, agent-aware robots, UA bot-block middleware
Cuts Vercel hobby edge-request burn on both skillkit (Vite static)
and skillkit-docs (Next.js fumadocs) projects. Site stays
human-first AND agent-first — Firecrawl, Context7, Crawl4AI,
OpenClaw, Hermes, ChatGPT-User, Claude-User, PerplexityBot
explicitly allowed; training crawlers and SEO scrapers blocked.

skillkit (Vite, docs/skillkit):
- vercel.json: Cache-Control on assets (1d browser, 7d edge,
  30d SWR; immutable for hashed /assets/*), HTML (5min browser,
  1d edge, 7d SWR), and /api JSON
- redirects with has user-agent: deflect known training+SEO
  bots to /robots.txt; negative lookahead on source prevents
  /robots.txt redirect loop
- public/robots.txt: explicit allow + deny lists

skillkit-docs (Next.js, docs/fumadocs):
- next.config.mjs: headers() block for /_next/static (1y
  immutable), assets (1d/7d/30d), /docs/* and / (5min/1d/7d)
- src/middleware.ts: UA-aware allow/deny pipeline; allowed
  agents pass through, blocked bots get 403 with cache header
  so Vercel edge serves the rejection cheaply
- public/robots.txt: same allow/deny list as marketing site
2026-04-28 11:16:20 +01:00
..