Categories
Categories:   All Categories
Suggested keywords
x
New

Blocked resources by robots.txt

JEvents (Free Access)
  JEvents (Free Access)
pl71
  pl71
  Friday, 17 April 2015
  3 Replies
  1.7K Visits
0
Votes
Undo
Shall I delete the following row in robots.txt:
Disallow: /components/

in order to suit Googlebot and to improve my SEO?

Googlebot complains that the following resource is blocked:
/components/com_jevents/views/flat/assets/css/modstyle.css?3.2.6

I noticed other developers are using media folder for assets/css files.
'media' folder is not disallowed in the latest versions of Joomla (after 3.3).
Oldest First  
tonyp
tonyp
Monday, 20 April 2015 10:19
·
#146558
0
Votes
Undo
Hello,

Which version of robots do you have? Joomla! did release an updated one.
Can you try adding:

Allow: /*.css$

Does that work?

Thanks
Tony
pl71
pl71
Monday, 20 April 2015 16:23
·
#146559
0
Votes
Undo
Latest Joomla!3.4.1:

User-agent: *

Disallow: /administrator/

Disallow: /bin/

Disallow: /cache/

Disallow: /cli/

Disallow: /components/

Disallow: /includes/

Disallow: /installation/

Disallow: /language/

Disallow: /layouts/

Disallow: /libraries/

Disallow: /logs/

Disallow: /modules/

Disallow: /plugins/

Disallow: /tmp/


Allow: /*.css$ is not working in my case.

So I've added: 
Allow: /components/com_jevents/views/flat/assets/css/

As per this article

Crawling the whole /components/ folder is not recommended, it is much better to separate css and js files in an allowed folder.
tonyp
tonyp
Monday, 20 April 2015 21:16
·
#146560
0
Votes
Undo
It is certainly much better to only allow certain files and folders. The above one I provided should have allowed it to access the .css files which are fine.

I'm glad you have resolved it nontheless.

Many thanks
Tony
  • Page :
  • 1
There are no replies made for this post yet.