user-agent: * Allow: / Allow: */cricket/test/ Allow: */cricketworldcup2019/Test/ Disallow: /toets/ Disallow: */toets/* Disallow: /test/ Disallow: */_test/* Disallow: */testpolar/* Disallow: */test/* Disallow: /xArchive/Archive/Illegal-liquor-export-20010319 Disallow: /.well-known/ Disallow: /assetlinks.json User-agent: Twitterbot Allow: / User-agent: ia_archiver Disallow: /BreakingNewsSms User-Agent: AhrefsBot Disallow: / User-Agent: MauiBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: ChatGPT-User Disallow: / ‍User-agent: anthropic-ai Disallow: / ‍User-agent: Claude-Web Disallow: / Sitemap: https://www.dieson.com/sitemap