Copy link to clipboard
Copied
Hi I am trying to find out how to remove the robots.txt file so I can index it with swiftype and use amuse themes widget to get a search function on my site so customers can search a long list of addresses by postcde (zipcode) anyone know how I can do this or is there a better way of building a search function into my site in muse ?
Copy link to clipboard
Copied
Find the file in file manager or via FTP and delete it.
Copy link to clipboard
Copied
I am having this same issue. I've looked in the file manager and FTP and do not see the robot.txt. However in the search results I get this error:
A description for this result is not available because of this site's robots.txt
I've checked the pages, Site Manager > Pages and all the pages say: Show this page for search engines
Your help is greatly appreciated.
Thank you.
Copy link to clipboard
Copied
Can you provide the link to your website and a screenshot of where you are seeing the message about robots.txt
Thanks
Copy link to clipboard
Copied
If you search on two brothers oshkosh in google it is the first listing. The site is twobrothersfamilyrestaurant.com.
Here is the search: Google
Please let me know if you need any other information. Thanks!
Copy link to clipboard
Copied
Not sure if it was Google or re-submitting the site map, but it's working now.
Thanks anyway.
Copy link to clipboard
Copied
Sitemap.xml is very different to robots.txt
Copy link to clipboard
Copied
Yes, I know that. Thank you. You have been extremely helpful!
Copy link to clipboard
Copied
Hello,
Ok, it is back again.
Any ideas why it keeps coming back?
The SEO Metadata, Search Results are all marked to Show this page for search engines.
There is not a robot.txt on the site that I can see via ftp or the file manager.
Any help is greatly appreciated. Thanks.
Copy link to clipboard
Copied
Hi
There is no robots.txt file on your site. try to open, yoursite.com/robots.txt, you will know what I mean.
You can create your own robots.txt and upload it to BC. To see the detailed instructions, follow this guide.
Let me know if you still have any question.
Copy link to clipboard
Copied
Thank you. I don't want a robot.txt on my site. I want the message: A description for this result is not available because of this site's robots.txt to be removed from the google search.
Google
Thank you.
Copy link to clipboard
Copied
If I search your site in Google, this is what I see,
Looks fine to me. Let me know if I am looking at wrong site.
Copy link to clipboard
Copied
That is not what I see unfortunately:
It was resolved for a while yesterday, but it came back.
Thank you for your help.
Copy link to clipboard
Copied
i do not see that either doing the same.
Try doing your search in incognito mode in a browser so you do not have cache, cookies or logged in.
Google search results are no longer the same for anyone by a long way.
Google search results runs all its calculations for lots of factors but if you are logged into google it will include your searching history to rank items based on that. Cookies and cache can also have an effect on your results.
Copy link to clipboard
Copied
Thank you. I was using incognito mode. I even tried on a computer that had never done the search.
I received the same results I posted above in both Google and Bing.
I will try to see if Google has an answer.
Thanks for your time.
Copy link to clipboard
Copied
Hi! I'm allowing myself to join the discussion (in 2018 ; ) ). I'm having the same issue, and google webmaster tools is showing me this message:
Is there a way i can change the noindex in the http-header in muse so that google can index my page?
Thank you for your time and help!
Copy link to clipboard
Copied
Login to the BC admin itself and just go into the page in question and ensure the SEO options are ticked to allow the page to be searched by google.