Google Index - sitemap not being fetched

Hello, we have been trying to index our marketplace for weeks now. We just cant get it to index on Google. We adapted the robots.txt, we checked the server etc. but it wont fetch and we have no idea why.

Can you check if this has happened with other marketplaces before or have further ideas (other than checking server again - 503)

We can see it, the robot.txt seems totally correct.

https://walanco.de/robots.txt

We do have a stage subdomain that was indexed before by itself (not sitemap provided) and tried to prevent Google from indexing thru https://stage.walanco.de/robots.txt

Also, the sitemap looks a bit chaotic and only includes categories. Not sure where we can create a proper sitemap that actually works, is structured and includes all products.

Any support would be highly appreciated.

In the robots.txt try to replace disallow line with

Allow: /

Yes, it looks like you’re blocking everything. The default robots.txt contains the following:

User-agent: *
Disallow: /app/
Disallow: /store_closed.html

Yes it is working now. We had SSL Certificate issues.

We are now trying to generate the sitemap but it will only grab the categories. Article are being ignored although activated

1 Like