By pl71 on Friday, 17 April 2015
Replies 3
Likes 0
Views 1.5K
Votes 0
Shall I delete the following row in robots.txt:
Disallow: /components/

in order to suit Googlebot and to improve my SEO?

Googlebot complains that the following resource is blocked:
/components/com_jevents/views/flat/assets/css/modstyle.css?3.2.6

I noticed other developers are using media folder for assets/css files.
'media' folder is not disallowed in the latest versions of Joomla (after 3.3).
Hello,

Which version of robots do you have? Joomla! did release an updated one.
Can you try adding:

Allow: /*.css$

Does that work?

Thanks
Tony
·
Monday, 20 April 2015 10:19
·
0 Likes
·
0 Votes
·
0 Comments
·
Latest Joomla!3.4.1:

User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/


Allow: /*.css$ is not working in my case.

So I've added:
Allow: /components/com_jevents/views/flat/assets/css/

As per this article

Crawling the whole /components/ folder is not recommended, it is much better to separate css and js files in an allowed folder.
·
Monday, 20 April 2015 16:23
·
0 Likes
·
0 Votes
·
0 Comments
·
It is certainly much better to only allow certain files and folders. The above one I provided should have allowed it to access the .css files which are fine.

I'm glad you have resolved it nontheless.

Many thanks
Tony
·
Monday, 20 April 2015 21:16
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post