# Gardener's Almanac — https://gardeners-almanac.app
#
# This file allows rather than restricts, on purpose. The plant catalogue and the garden book
# are server-rendered, bilingual public pages built to be read, indexed and freely scraped —
# see docs/decisions/features/plant-publicity.md, which calls that knowledge a public good and
# notes there is no business reason to hide it — and since a permissive robots.txt is a
# statement about crawling rather than about rights, the text on those pages is licensed
# CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), which is the part that actually
# lets you keep what you fetched. What the pages do not carry, and what is not licensed, is
# the cultivation figures behind each plant: they are the calibration the timing calculations
# are tuned against rather than advice, and CONTENT-LICENSE.md and
# docs/decisions/features/cultivation-figures-are-internal.md say so at length.
#
# AI crawlers are covered by the wildcard below and are meant to be: they are named nowhere
# here because allowing them needs no rule, and a list of agents is a list somebody has to
# maintain forever.
#
# What is disallowed is only what crawling cannot usefully reach. Everything under /api/,
# /oauth/ and /mcp answers 401 or a protocol document to anything without a session, and
# /internal/ is never proxied to the internet at all. The app's own routes are listed for a
# different reason: the SPA answers 200 with the application shell for any path it does not
# recognise, so without these lines every signed-out crawl of /gardens/anything would index
# the same empty page under a different URL.
#
# The app-route list below mirrors the routes in frontend/src/App.tsx, and since
# signed-out-shell.md phase 5 something enforces that: internal/server/robots_routes_test.go
# reads both files and fails when a route in App.tsx is not classified, or is classified one
# way here and another there. That test carries the reason for each decision, including the
# two routes deliberately left crawlable.
#
# The check was worth building at that phase because it is when /plants and /knowledge began
# rendering real content to a crawler that executes JavaScript, rather than the empty shell
# a signed-out crawl used to get. An unlisted route stopped being merely a duplicate of the
# shell and became a second, uncanonical copy of a page we publish on purpose.
#
# Served straight out of frontend/public/, by nginx in production and by Vite in dev, so it
# needs no route, no location block and no dev-proxy entry. The sitemaps do — they are built
# from the database and the article corpus. See internal/server/seo_handlers.go.

User-agent: *
Disallow: /api/
Disallow: /oauth/
Disallow: /internal/
Disallow: /mcp
Disallow: /gardens/
Disallow: /knowledge
Disallow: /settings
Disallow: /feedback
Disallow: /login
Disallow: /consent
Disallow: /invite
Allow: /

# Absolute, and naming production deliberately: this file ships in every image, and a crawler
# reading it from anywhere else should still be pointed at the site that is meant to be indexed.
Sitemap: https://gardeners-almanac.app/sitemap.xml