Setting up robots.txt for a bakery

Automate robots.txt if you can. If you cannot, write a quarterly reminder in a calendar that someone actually checks. an inbox shared between the office and the webmaster is what happens when nobody does either.

The hosting bill for a bakery is twelve dollars a month. Their old provider charged eighty and included a page builder nobody used. robots.txt was broken on both plans.

When we reviewed robots.txt, the DNS panel showed an inbox shared between the office and the webmaster. Two records contradicted each other, and the resolver picked one at random.

We document robots.txt in a shared spreadsheet. It is not glamorous, but a bakery can open it on their phone at midnight if something breaks.

The job started when a bakery realised their robots.txt had been wrong for months. Nobody noticed because the site still loaded.

We check robots.txt for a bakery once a quarter. It takes five minutes and has caught two problems in the past year. Both would have caused downtime within a week.

We hand over a one-page document after every robots.txt job. It says what was changed, what the login is, and what to check if something feels wrong. a bakery keeps it in a drawer they actually open.

More notes

Related notes

• Setting up robots.txt for a small hotel
• What a hardware store needs to know about robots.txt
• Managing robots.txt at a florist
• A practical look at robots.txt for a plant nursery