Boas Pessoal,
Estive a renovar o meu file robots.txt que tenho nos sites em wordpress e ficou assim:
User-agent: *
# disallow all files in these directories
Disallow: /cgi-bin
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback/
User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$
# Google Image
User-agent: Googlebot-Image
Allow: /
# Google AdSense
User-agent: Mediapartners-Google
Allow: /
# Internet Archiver Wayback Machine
User-agent: ia_archiver
Disallow: /
# digg mirror
User-agent: duggmirror
Disallow: /
Gostaria de obter algumas opiniões de users mais experientes sobre o conteudo do meu file robots.txt atrás referido.
Gracias