Add
[Content-Signal](https://www.ietf.org/archive/id/draft-romm-aipref-contentsignals-00.txt)
directive to robots.txt opting into AI training, search, and inference
use.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated website and documentation robots.txt to enable AI training,
search indexing, and AI input processing for site content.
* Minor formatting fix: ensured trailing newline at end of both
robots.txt files.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->