Help with robots.txt

[CK13]

This is my robots.txt

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/

What does that mean?
Is everything ok, do I have to add somethig else to it?

PS: Still learning :P .

 

[Rox]

add ur sitemap and rss feeds

Example below:

Sitemap: http://www.yoursite.com/sitemap.xml
Sitemap: http://www.yoursite.com/category/movies/feed/rss
Sitemap: http://www.yoursite.com/category/tv-shows/feed/rss
Sitemap: http://www.yoursite.com/category/movies/exclusive-encodes/feed/rss

 

[Daniel]

Here's an example that is often used

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Sitemap: http://www.yoursite.com/sitemap.gz

What you should do
Create a sitemap and add it to the bottom of the text file.

 

[CK13]

@ MrSandvik If I put you're code will this be allowed:
[slide]http://imgcrave.com/u/untitlefe.jpg[/slide]

And is it safe to remove what is already there?

 

[Daniel]

You can easily use such a generator. No problem.

That's why they exist.

 

[CK13]

But what about my current robots file should i replace it or just copy the additional code?
Is it safe to remove what is already there?

 

[Vince(AdJoin.Me)]

Here's an example that is often used

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Sitemap: http://www.yoursite.com/sitemap.gz

What you should do
Create a sitemap and add it to the bottom of the text file.

you are helping him to do the right thing.