Upload Your Robots.txt File
If you choose to create a robots.txt file in an external file, you must upload the file to the correct location on your Salesforce B2C Commerce server.
Users need to upload and manage this file via UX Studio in the cartridge/static/default directory in their custom storefront cartridge.
You can also manually update the robots.txt file in production after a staging process if the file was accidentally overwritten.
There can only be a single robots.txt file per cartridge. Specifically, don't put robots.txt files in user directories, because a robot never looks at them.
Note: The robots.txt file can only be replicated from instance
to instance via code replication. This is because the
cartridge/static/default is cartridge-specific not
site-specific.