What is a sitemap? (and why search engines like one)
A sitemap is a list of your store's pages for search engines, and a robots file tells them where not to go; here is what each does and how Varchas handles them.
A sitemap in plain terms
Think of a sitemap as a contents page for your store, written for search engines instead of people. It is a file that lists the pages you want found: your home page, your shop page, each product and so on. A search engine can read it and know what exists without wandering around hoping to find every link.
A sitemap does not guarantee that your pages will appear in search results, and it does not make them rank higher. It only helps search engines discover them. That is useful for a new store that has few links pointing at it from elsewhere.
The robots file
The robots file is the other half. It is a short instruction note placed at a standard address on your site. It tells well-behaved crawlers which areas they should not bother with. A shopper's bag, order pages and admin screens are good examples, since they are useless in a search result.
Be clear on one thing: a robots file is a polite request, not a lock. It is not a way to protect private information. Anything truly private must be protected by login, not by a note.
What Varchas does
From the code behind the storefront, a Varchas store generates both files automatically, so you do not write them by hand.
- The sitemap is generated for each store and lists that store's own public pages and its live products. It is built from your current catalogue rather than a fixed list, so a product you publish or archive is reflected without you editing anything.
- Filtered shop views, such as a particular colour and size, are deliberately left out, because they repeat products that are already listed.
- The robots file points crawlers to the store's sitemap and asks them to stay out of shopper-specific pages, such as the bag and account, and out of the admin portal.
- Before a store is open, these are set up so that nothing is advertised that would only redirect.
That means your main job is not to maintain the files. It is to keep your products published with clear titles and descriptions, which is what the sitemap points search engines towards.
Where you can check it
Most sites place the sitemap at the address of your site followed by /sitemap.xml and the robots file at /robots.txt. Open them in a browser on your own domain and look at what they show. If your products are there, the setup is working. If your store uses its own domain, see your brand on its own domain for why a consistent address matters.
What to do after
- Give every product a clear title. How to write a product title that gets found is a good starting point.
- Write useful descriptions, as in writing product descriptions people read.
- If you use Google's free search tool for site owners, you can submit your sitemap address there. Follow that tool's current instructions, since screens change.
Common mistakes
- Expecting the sitemap to bring instant traffic. It takes time for search engines to visit and decide.
- Blocking pages by accident when editing a robots file by hand on another platform.
- Leaving products in draft, so there is nothing for the sitemap to list.
Where to start
Open your own sitemap link today. Count the products listed and compare with what you intended to publish. Any missing product is worth checking.