User-agent: * Disallow: / User-agent: Screaming Frog SEO Spider Disallow: User-agent: SEMrushBot Disallow: User-agent: SemrushBot-SA Disallow: