Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add per-site well-known files: sitemap, robots.txt, and dashboard editors#12923

Open
wordish wants to merge 2 commits into
concretecms:9.5.xfrom
wordish:feature/multisite-well-known-files
Open

Add per-site well-known files: sitemap, robots.txt, and dashboard editors#12923
wordish wants to merge 2 commits into
concretecms:9.5.xfrom
wordish:feature/multisite-well-known-files

Conversation

@wordish
Copy link
Copy Markdown
Contributor

@wordish wordish commented May 21, 2026

Adds a Well-Known Files dashboard page (System > SEO & Statistics) where admins can edit per-site robots.txt, llms.txt, security.txt, ads.txt, and humans.txt. In multisite installs files are stored under application/files/site-specific/{host}/ and served per-domain via nginx/Apache routing rules shown in the dashboard. In single-site installs files are written directly to the webroot — no server config changes needed.

The Generate Sitemap task writes sitemap.xml to the correct location for the install mode. NginxGenerator and ApacheGenerator include the well-known routing rules with hostname validation to prevent path traversal via the Host header. Dashboard save actions use atomic writes, enforce CSRF, bound content to 64 KB, and check per-site admin permission.

Addresses #11176: per-site sitemap and well-known files in multisite

Reported-by: deanL-zuiderlicht and KurtKeunen

Prior art: hissy built
MacareuxDigital/md_multisite_sitemap as a community workaround for the
same problem. That package's existence demonstrated the real-world demand
and informed the decision to address this in core.

Prior art: hamzaouibacha proposed #12640 to deal with the same problem
which provided another starting point for this solution.

Thanks to HamedDarragi, mlocati, KorvinSzanto, and aembler for insights
and discusison that also informed decisions in this PR.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 21, 2026

CLA assistant check
All committers have signed the CLA.

@wordish wordish force-pushed the feature/multisite-well-known-files branch from 86f022d to ad43d7b Compare May 21, 2026 10:51
…tors

Adds a Well-Known Files dashboard page (System > SEO & Statistics) where
admins can edit per-site robots.txt, llms.txt, security.txt, ads.txt, and
humans.txt. In multisite installs files are stored under
application/files/site-specific/{host}/ and served per-domain via
nginx/Apache routing rules shown in the dashboard. In single-site installs
files are written directly to the webroot — no server config changes needed.

The Generate Sitemap task now writes sitemap.xml to the correct location
for the install mode. NginxGenerator and ApacheGenerator include the
well-known routing rules with hostname validation to prevent path traversal
via the Host header. Dashboard save actions use atomic writes, enforce CSRF,
bound content to 64 KB, and check per-site admin permission.
@wordish wordish force-pushed the feature/multisite-well-known-files branch from ad43d7b to 8b42b39 Compare May 21, 2026 10:55
- Fix t() output: move HTML out of translated strings into sprintf %s arguments; add h() on heading output
- Fix base64_decode silent failure: early return on invalid encoding instead of writing empty file
- Fix Apache well-known RewriteCond double-slash: use $1 (filename capture) not %{REQUEST_URI}
- Fix WellKnownFileManager static methods: convert to instance methods so the class is replaceable via the container; fix same issue in GenerateSitemapCommandHandler and GenerateRobotsTxtCommandHandler
- Add filename allowlist (ALLOWED_FILENAMES) to writeFile(), getFilePath(), and getFilePathForSite() to prevent path traversal
- Add realpath() confinement in writeToPath() as defense-in-depth
- Add test coverage: getFilePath() paths, disallowed filename guards, Apache/Nginx well-known rule assertions

Resolve DI and form encoding issues in well-known files dashboard

- Replace new NginxGenerator() / new ApacheGenerator() with $this->app->make() so the container is used and the classes remain replaceable
- Remove base64 encode/decode round-trip: give each editor textarea name="content" directly and drop the hidden input and submit-time JS; standard form encoding handles multiline text fine

Fix multisite threshold and tighten WellKnownFileManager API

Tests cover all four canonical-count threshold cases: zero sites,
one site without canonical URL, two sites with only one configured,
and two sites fully configured.

Fix i18n hygiene and @Unlink suppressors

Preserve per-site robots.txt custom rules on sitemap regeneration

Clarify that multisite server config rules are required, not optional

Addresses concretecms#11176: per-site sitemap and well-known files in multisite

Reported-by: deanL-zuiderlicht and KurtKeunen

Prior art: hissy built
MacareuxDigital/md_multisite_sitemap as a community workaround for the
same problem. That package's existence demonstrated the real-world demand
and informed the decision to address this in core.

Prior art: hamzaouibacha proposed concretecms#12640 to deal with the same problem
which provided another starting point for this solution.

Thanks to HamedDarragi, mlocati, KorvinSzanto, and aembler for insights
and discusison that also informed decisions in this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants