I understand that to restrict the web search robots from indexing a folder, I should specify this in a robots.txt file. My question is: Where do I put this file?
I'm told to put it in the root area, but as I have several domains pointing to a subfolder, I'm confused about what this means. Is it the root folder of my Bravenet area - and if so, where is that? There is no "root" folder, but a "web" folder. In the web folder I have five subfolders, each containing a separate site. The folder I want the search robots not to index is a subfolder to one of these sites.
Or is it as simple as putting the robots.txt file in the folder containing the index.html page of the particular site? This of course being where the domain name points to.
YOu keep all main pages in your root area or main area of your account.
Then make folders for things like pictures and music or whatever else it helps keep stuff organized with your site so everything isn't all cluttered together.
Each domain or sub-domain can have only one robots.txt file which goes in the same folder as the index.html file that displays when that domain name is typed in without specifying a filename. The robots/spiders will look there for the file regardless of where they enter the site from.
My robots.txt file contains the following lines to stop the image and include folders from being searched.
