#39094 left the model profile refresh workflow invalid, so it failed
immediately on push with no jobs (e.g.
https://github.com/langchain-ai/langchain/actions/runs/30287294147).
---
GitHub Actions expressions only allow single-quoted string literals. The
new job condition used double quotes:
```yaml
if: github.repository_owner == "langchain-ai"
```
That failed workflow validation before any job could start:
> Unexpected symbol: `"langchain-ai"`
Also switched the hyphenated input to bracket access so
`providers-override` is not parsed as subtraction (`inputs.providers -
override`).
Manual runs of the model profile refresh workflow can now target
specific providers via a dropdown or comma-separated override; the
scheduled daily run is unchanged and still refreshes all providers.
---
When manually running the model profile refresh workflow, you previously
always refreshed every in-monorepo provider. That made quick,
single-provider updates noisy: the bot PR mixed unrelated providers,
took longer, and was harder to review.
This adds a `workflow_dispatch` filter so maintainers can refresh one or
more providers on demand, while the daily schedule still covers
everyone.
- **Dropdown** `provider`: `all` (default) or a models.dev provider ID
- **Free-text** `providers-override`: comma-separated IDs that take
precedence over the dropdown (package aliases `fireworks` and
`mistralai` are accepted)
- Filtered runs use a tighter PR title/body, e.g.
`chore(model-profiles): refresh openai model profile data`
- Unknown provider IDs fail fast before refresh starts
`build_summary` in `langchain_model_profiles._summary` now wraps each
provider section in a `<details>` toggle when more than one provider has
changes, making multi-provider refresh PR summaries skimmable.
Single-provider summaries are unchanged.
---
When the `refresh_model_profiles` workflow refreshes data for multiple
partner integrations at once, the resulting PR summary can be long and
hard to skim — every provider's added/removed/changed rows are rendered
flat, one after another. This wraps each provider section in a
`<details>`/`<summary>` toggle when more than one provider has changes,
so reviewers can expand only the providers they care about.
Single-provider summaries stay flat since there's nothing to collapse.
The per-provider `### {provider}` heading is stripped inside toggles so
the `<summary>` tag is the sole label — no duplicated provider name.
Also includes two smaller changes that were staged alongside:
- The `refresh_model_profiles` and `_refresh_model_profiles` workflow
PR-body templates now link to the workflow file instead of bare-text
referencing it.
- `AGENTS.md` and `CLAUDE.md` document the LangSmith integration test
tracing setup: the env vars CI sets, the pytest plugin that bridges
`LANGSMITH_TAGS`/`LANGSMITH_METADATA` into `tracing_context`, and the
unit-test isolation approach.
---------
Signed-off-by: Mason Daugherty <github@mdrxy.com>
Co-authored-by: open-swe[bot] <215916821+open-swe[bot]@users.noreply.github.com>
Closes#37997
Forked repositories with Actions enabled currently run the scheduled
model profile refresh without access to the GitHub App secrets used to
open the automated PR. Guarding the job to the `langchain-ai` owner
prevents noisy daily failures on forks while preserving the scheduled
refresh for the main repository.
## Changes
- Added a repository-owner guard to the `refresh-profiles` job so
`refresh_model_profiles` only runs under `langchain-ai`.
- Kept the existing reusable workflow invocation and bot secret wiring
unchanged for the canonical repository.
Extract the model profile refresh logic into a reusable `workflow_call`
workflow so external repos like `langchain-google` and `langchain-aws`
can run the same daily profile refresh and get auto-PRs without
duplicating the pipeline. The in-monorepo caller becomes a thin wrapper
passing provider JSON.
## Changes
- Add `_refresh_model_profiles.yml` as a reusable `workflow_call`
workflow — accepts a `providers` JSON array of `{provider, data_dir}`
pairs, optional `cli-path` (skips cloning the CLI repo when the caller
already has it), and configurable PR metadata inputs
- External callers get the `langchain-profiles` CLI via sparse checkout
of `langchain-ai/langchain` at a configurable `cli-ref`; the in-monorepo
caller short-circuits with `cli-path: libs/model-profiles`
- Add input validation step using `jq` — rejects non-array JSON and
entries missing `provider`/`data_dir` keys with `::error::` annotations
- Replace the piped `while read` loop with `mapfile`/`for` +
per-provider error handling: one provider failure no longer kills the
rest, and all failures are collected and reported at the end
- Route all `${{ inputs.* }}` expressions through `env:` bindings in
`run:` blocks to prevent script injection from caller-controlled values
- Validate `cli-path` existence before use, with a clear error if the
directory is missing
- Summary step now runs with `if: always()` and handles
failure/success/no-op states separately
- Refactor `refresh_model_profiles.yml` into a thin caller that passes
the 10 in-monorepo providers as JSON
Extract additional fields from models.dev into `_model_data_to_profile`:
`name`, `status`, `release_date`, `last_updated`, `open_weights`,
`attachment`, `temperature`
Move the model profile refresh logic from an inline bash script in the
GitHub Actions workflow into a `make refresh-profiles` target in
`libs/model-profiles/Makefile`. This makes it runnable locally with a
single command and keeps the provider map in one place instead of
duplicated between CI and developer docs.
## Summary
- Adds top-level `permissions: contents: read` to 5 workflows that only
had job-level permissions: `pr_labeler_file`, `pr_labeler_title`,
`tag-external-contributions`, `v03_api_doc_build`,
`auto-label-by-package`
- SHA-pins all 14 third-party actions to full commit SHAs to prevent
supply chain attacks via tag hijacking
## Why
**Missing top-level permissions:** Without an explicit top-level
`permissions` block, workflows inherit the repository/org default token
permissions, which may be overly broad. Adding `contents: read` as the
default restricts the blast radius if a dependency or action step is
compromised.
**SHA pinning:** Mutable tags (`@v1`, `@master`) can be force-pushed by
the action maintainer or an attacker who compromises their account.
Pinning to a full 40-character SHA ensures the exact reviewed code
always runs. Tag comments are preserved for readability.
### Actions pinned
| Action | File(s) |
|--------|---------|
| `pypa/gh-action-pypi-publish` | `_release.yml` (2 uses) |
| `ncipollo/release-action` | `_release.yml` |
| `Ana06/get-changed-files` | `check_diffs.yml` |
| `astral-sh/setup-uv` | `check_diffs.yml`, `uv_setup/action.yml` |
| `CodSpeedHQ/action` | `check_diffs.yml` |
| `google-github-actions/auth` | `integration_tests.yml` |
| `aws-actions/configure-aws-credentials` | `integration_tests.yml` |
| `amannn/action-semantic-pull-request` | `pr_lint.yml` |
| `bcoe/conventional-release-labels` | `pr_labeler_title.yml` |
| `mikefarah/yq` | `v03_api_doc_build.yml` |
| `EndBug/add-and-commit` | `v03_api_doc_build.yml` |
| `peter-evans/create-pull-request` | `refresh_model_profiles.yml` |
## Test plan
- [x] CI passes — all workflows still resolve their actions correctly
- [x] Verify no functional change: SHA refs point to the same code as
the previous tags
---
> This PR was generated with assistance from an AI coding agent as part
of a repository posture check.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Schedules the `refresh_model_profiles` workflow to run daily at 08:00
UTC (manual trigger available).
- Adds a job summary step that reports whether a PR was created/updated
or skipped because profiles were already up to date.
- Each run supersedes any stale PR from a previous run since the action
force-pushes to a fixed branch (`bot/refresh-model-profiles`).
Mason Daugherty