0 Fork 0

mirror of https://github.com/langchain-ai/langchain.git synced 2026-10-05 09:25:14 +03:00

Files

T

github-actions[bot]andnpentrel 42f04f460d docs: update OpenWiki (#40781 )

Automated OpenWiki documentation update.

OpenWiki result: success

When the result is `failure`, this PR intentionally preserves only the
pages completed before the failure. Merge it to make that progress the
baseline for the next scheduled run.

Co-authored-by: npentrel <5212232+npentrel@users.noreply.github.com>

2026-09-28 09:57:48 -04:00

20 KiB

Raw Permalink Blame History

type, title, description, tags, verified, sources, generated

type

title

description

CI/CD Workflows: GitHub Actions and Release Process

LangChain employs a sophisticated CI/CD system built on GitHub Actions that automates testing, linting, quality checks, and release management across a monorepo structure. The system emphasizes efficiency through intelligent change detection, parallel matrix testing, and strict release gates.

Architecture Overview

The CI/CD system consists of three layers:

Pull request / push CI (check_diffs.yml): Detects changed packages and runs targeted tests, linting, and compatibility checks
Scheduled integration testing (integration_tests.yml): Daily remote API testing with live credentials against partner libraries
Manual release workflow (_release.yml): Comprehensive pre-release validation, PyPI publishing, and dependent package testing

Primary CI Workflow (Pull Requests & Master Pushes)

The main entry point is .github/workflows/check_diffs.yml, which runs on every pull request, push to master, and merge group event.

Change Detection & Matrix Generation

The workflow begins with a change detection phase:

A Python script (.github/scripts/check_diff.py) analyzes which files changed
Maps changes to package directories (libs/core, libs/partners/*, etc.)
Builds a dependency graph to include dependent packages when core components change
Generates separate test matrices for linting, unit tests, Pydantic compatibility tests, integration test compilation, VCR cassette tests, and extended test suites
Outputs are passed as JSON to downstream jobs via matrix strategy

This detection ensures only affected packages are tested, optimizing CI runtime. The script skips the libs/standard-tests directory in enumeration and treats certain partners (e.g., huggingface) as CI-unstable, removing them from dependent chains while allowing direct edits to be tested.

Linting Pipeline (`_lint.yml`)

Runs on affected packages with Python 3.11 (configurable):

Ruff analysis: Code style, import sorting, and rule enforcement with inline GitHub annotations (via RUFF_OUTPUT_FORMAT: github)
MyPy type checking: Static type verification
Markdown linting: Documentation quality checks (via .markdownlint.json)

Tools are sourced from dependency groups: lint and typing. The workflow installs both package code and test code dependencies, running make lint_package and make lint_tests targets. Partner packages receive separate test dependency installation including integration test dependencies.

Unit Testing (`_test.yml`)

Runs matrix tests across Python versions with dependency constraint verification:

Matrix dimensions:

Python 3.10, 3.11, 3.12, 3.13, 3.14 for libs/core
Python 3.10 and 3.14 for other packages
Current locked dependencies (from uv.lock)
Minimum supported dependency versions

Two-phase testing:

Current dependencies: Runs full test suite against versions in uv.lock via make test PYTEST_EXTRA=-q
Minimum dependencies: Calculates minimum versions from pyproject.toml constraints via get_min_versions.py script, downgrades via pip, and reruns tests with make tests PYTEST_EXTRA=-q to ensure compatibility

The workflow verifies the working directory remains clean (no untracked generated files) after testing.

Pydantic Compatibility Testing (`_test_pydantic.yml`)

Tests affected packages against configurable Pydantic versions (e.g., v2.0, v2.1, v2.2):

Triggered when Pydantic version constraints or dependent code changes
Determines test matrix by querying uv.lock for max Pydantic version and pyproject.toml for min, across both core and target packages
Runs make test against each Pydantic version
Uses Python 3.12 by default with override support

VCR Cassette Tests (`_test_vcr.yml`)

Validates integration tests backed by recorded HTTP cassettes:

Runs in playback-only mode with fake credentials (no real API keys required)
Detects stale cassettes from test input changes without re-recording
Executes make test_vcr target
Enables fast, repeatable integration test feedback

Only triggered for packages with VCR cassettes (currently libs/partners/openai), as tracked in the VCR_PACKAGES set within check_diff.py.

Integration Test Compilation (`_compile_integration_test.yml`)

Performs shallow integration test validation:

Compiles test modules without executing them via pytest -m compile tests/integration_tests
Catches import errors and obvious syntax issues
Provides quick feedback loop without running expensive external API calls
Installs both test and integration test dependency groups

Extended Test Suites

For packages defining extended_testing_deps.txt, runs additional tests:

Installs extra dependencies beyond standard test group via the file
Executes make extended_tests target
Allows performance benchmarks, stress tests, or heavy-weight validations
Located in the extended-tests job within check_diffs.yml

Release Option Validation

The workflow includes a check-release-options job:

Verifies .github/workflows/_release.yml dropdown options stay synchronized with actual package directories
Prevents stale release options from blocking valid releases

Release Workflow (`_release.yml`)

The release workflow is manually triggered via GitHub Actions UI (or can be called as a reusable workflow). It handles versioning, building, testing, and publishing to PyPI.

Release Modes & Invocation

Manual dispatch (workflow_dispatch):

Dropdown selection of package to release (core, langchain, langchain_v1, text-splitters, standard-tests, model-profiles, or 17+ partner packages)
Manual version entry (default 0.1.0)
Optional override to full path (e.g., libs/partners/partner-xyz)
Dangerous flags: dangerous-nonmaster-release (hotfixes), allow-prereleases, skip-prior-published-package-checks

Reusable workflow (workflow_call):

Accepts working-directory, release-version, and safety bypass flags
Used internally for multi-package release orchestration

Release Gate: Build & Version Check

Job: build (isolated permissions for security):

Version verification: Extracts version from pyproject.toml and compares against input using PEP 440 normalization (treating 0.1.0-rc1 and 0.1.0rc1 as equivalent); fails if mismatch
PyPI availability check: Queries https://pypi.org/pypi/{pkg}/{version}/json to ensure version not already published; fails closed if PyPI is unreachable or returns unexpected status
Build: Runs uv build to create wheel and sdist distributions
Artifact upload: Stores dist/ directory for downstream jobs

Security rationale: Separates build (no credentials) from publishing (trusted publishing token) to prevent compromised dependencies from accessing PyPI credentials.

Release Notes Generation

Job: release-notes:

Tag detection: Finds previous release tag via git history
- For pre-releases (contains hyphen): Matches base version; falls back to latest release tag
- For stable releases: Searches for previous patch version; falls back to latest
- First release: Uses full commit history from git root
Changelog extraction: Runs git log --format="%s" <prev-tag>..HEAD -- <working-dir> to collect commit messages
Tag validation: Confirms previous tag exists in git repo before proceeding

Pre-Release Checks

Job: pre-release-checks (no caching to catch missing dependencies):

Direct wheel installation: Installs built wheel directly via uv pip install dist/*.whl (validates metadata and installability)
Package import test: Verifies main module imports successfully
Unit tests: Runs full make tests against the wheel
Minimum version testing: Recalculates minimum versions, downgrades via pip, and reruns tests with make tests PYTEST_EXTRA="-q -k 'not test_serdes'" (skips serialization tests for speed)
Prerelease dependency detection: Fails if any dependencies declare prerelease versions (unless release itself is prerelease)
Integration tests: For partner packages only, runs make integration_tests with live API credentials

PyPI Publishing

Job: test-pypi-publish (TestPyPI):

Uses GitHub OpenID Connect (trusted publishing)
Publishes to test.pypi.org for staging validation
Tolerates duplicate versions via skip-existing: true (CI safety only)

Job: publish (Production PyPI):

Uses trusted publishing to production PyPI
Only runs if all prior checks pass
Creates GitHub Release with generated release notes

Compatibility Testing

Job: test-prior-published-packages-against-new-core:

Only runs for libs/core releases
Tests previously-published partner packages (currently anthropic, openai) against new core
Fetches latest non-yanked published partner tag from git, installs new core wheel, runs tests
Can skip per-partner via skip-prior-published-package-checks input (options: none, anthropic, openai, all)

Job: test-dependents:

Only runs for libs/core or libs/langchain_v1 releases
Checks external dependent packages (currently deepagents)
Tests Python 3.11 and 3.13
Ensures breaking changes are caught before publish

Integration Testing (`integration_tests.yml`)

Scheduled daily (1 PM UTC) with manual dispatch override capability.

Test Matrix Generation

Job: compute-matrix:

Default scope: Tests 9 partner libraries (OpenAI, Anthropic, Fireworks, Groq, MistralAI, XAI, Google VertexAI, Google GenAI, AWS)
Python versions: 3.10 and 3.14 by default; overridable via input
Selective testing: Can select single library, exclude libraries, or override Python versions
Scope security: Only runs on main repository; manual dispatch allowed from forks

Integration Test Execution

Job: integration-tests:

Checks out primary monorepo plus external google-genai, google-vertexai, and langchain-aws repositories
Reorganizes external repos into local partner directories for unified testing
Authenticates to Google Cloud and AWS
Runs per-package make integration_tests with all live API credentials injected
Uses concurrency locks per (package, python-version) to serialize same-package runs and prevent credential conflicts
Includes special installation logic: overlays local editable core and standard-tests packages atop checked-out partner versions

Credentials: Receives 30+ environment variables covering OpenAI, Anthropic, Google, AWS, Azure, Groq, MistralAI, HuggingFace, Mistral, Together, Cohere, and more.

Auto-Labeling Workflows

Issue Auto-Labeling (`auto-label-by-package.yml`)

Fires when issues are opened or edited:

Parses issue body for ## Package section
Supports both dropdown (single select) and checkbox (multi-select) formats
Maps package name (e.g., "langchain-openai") to label via JSON mapping table (e.g., "openai")
Adds/removes labels to match selected package(s)

PR Title Linting (`pr_lint.yml`)

Enforces Conventional Commits 1.0.0 format on all pull request titles:

Format: <type>[optional scope]: <description> (e.g., feat(core): add multi-tenant support)
Allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert, release, hotfix
Optional scope: Scopes for specific packages (core, langchain, anthropic, openai, etc.) or cross-cutting concerns (infra, deps, partners)
Breaking changes: Append ! after type/scope (e.g., feat!: remove deprecated API)
Release commits: Must be release(scope): x.y.z format
Validation: Uses amannn/action-semantic-pull-request with empty scope rejection

Empty scope parentheses are rejected; PR must either omit parentheses (no scope) or provide a valid scope.

PR Labeling (`pr_labeler.yml`)

Unified PR labeler applying size, file-based, title-based, and contributor classification:

File-based labels: Maps changed file paths to package labels
Size labels: Computes PR size (small, medium, large) from diff statistics
Title-based labels: Detects certain patterns in PR title
Contributor classification: Checks org membership to tag external contributions (via GitHub App token)
Uses concurrency locks to prevent race conditions
Consolidates multiple prior workflows into single sequential run

OpenWiki Auto-Update (`openwiki-update.yml`)

Runs on schedule (8 AM UTC daily) or manual dispatch:

Checks out full repository history via fetch-depth: 0 (required for diff-against-HEAD)
Installs Node.js and OpenWiki CLI (@0.5.0) with optional Mermaid diagram validation
Runs openwiki code --update --print to regenerate documentation
Removes transient state file (.run.json)
Creates/updates pull request with changes via peter-evans/create-pull-request@v8.1.1
Preserves partial progress on failure: if OpenWiki run fails, the PR intentionally preserves only pages completed before the failure, allowing them to become baseline for the next scheduled run

Uses LangSmith tracing for observability (OPENWIKI_LANGSMITH_API_KEY, LANGSMITH_API_KEY).

Dependency Pinning & Version Management

Frozen Dependency Locks

All CI jobs set UV_FROZEN=true and UV_NO_SYNC=true (when applicable):

Ensures reproducible builds against locked versions in uv.lock
Prevents transitive dependency surprises in CI
Each job explicitly pins Python version and dependency revisions

Minimum Version Testing

The get_min_versions.py script extracts version constraints from pyproject.toml and queries PyPI for minimum published versions satisfying those constraints.

Example: If constraint is langchain-core>=0.3.0,<1.0, the script finds and installs the earliest 0.3.* release.

Two modes:

pull_request: Tests against minimum with some leniency (used in PR CI)
release: Stricter testing with prerelease rejection (used in release validation)

Release Policy

Semantic Versioning

Core (libs/core) follows strict semantic versioning:

Major version: Breaking changes
Minor version: New features (backward compatible)
Patch version: Bug fixes

Partner packages and other libraries align with core releases:

LangChain follows core versioning for tight integration
Partners maintain independent versioning but coordinate with core releases

Release Branching

Releases only proceed from master branch (default) or explicitly via dangerous-nonmaster-release flag (hotfixes only)
Version must match pyproject.toml or operator provides override
PyPI availability double-checked to prevent accidental re-publishes

Pre-Release Support

Supports alpha/beta/rc versions (e.g., 0.1.0-rc1, 0.1.0a1)
Pre-release detection normalizes hyphen/underscore variants per PEP 440
Optional allow-prereleases flag permits transitive prerelease dependencies during alpha cycles
Final releases block prerelease dependencies unless explicitly allowed

Configuration & Operations

Environment Variables

Frozen dependency control:

UV_FROZEN: Prevents automatic dependency resolution
UV_NO_SYNC: Skips uv sync in build steps (manual sync used instead)

Linting & formatting:

RUFF_OUTPUT_FORMAT: github: Inline GitHub annotations for linter violations

LangSmith tracing (optional):

LANGSMITH_API_KEY: Optional tracing of CI workflows themselves
LANGCHAIN_TRACING_V2: true: Enable tracing
LANGCHAIN_PROJECT: openwiki: LangSmith project name

GitHub Actions Permissions

Workflows follow principle of least privilege:

Default: contents: read (read-only)
PR labeler: pull-requests: write, issues: write
Release: id-token: write (trusted publishing), contents: write (GitHub Release creation)
OpenWiki update: contents: write, pull-requests: write

Isolated jobs (build, testing) receive no write permissions; publishing jobs run in separate jobs with restricted scope.

Custom Actions

uv_setup (.github/actions/uv_setup):

Sets up Python via official setup-python action
Configures uv tool with optional caching
Supports per-package cache suffixes to avoid cross-contamination
Parameters: python-version, cache-suffix, working-directory, enable-cache

Important Invariants & Failure Modes

No caching in release pre-checks: Missing dependencies would be masked by cached venvs, allowing broken releases to publish
Minimum version downgrade isolation: Minimum version tests reinstall packages in fresh virtual environment context, not via constraint relaxation alone
Separate build/publish jobs: Build job has no PyPI credentials; publishing job has no build tools, preventing supply-chain attacks
Change detection scope: VCR and extended test matrices only include packages with appropriate markers; adding test files without markers won't trigger corresponding test suites
Prerelease blocking: Stable releases reject any prerelease dependencies, preventing version resolution issues in downstream users
Tag/version synchronization: Release workflow validates git tags match expected version format before publishing, catching manual tag drift

Extension Points

Adding new package types: Update check_diff.py to recognize new directories and map them to appropriate test matrices
Adding partners to release testing: Update test-prior-published-packages-against-new-core matrix and skip-prior-published-package-checks input options (keep in sync)
Adding new linting/type checkers: Extend _lint.yml job steps and dependency groups; ensure make lint_package target exists
Adding integration test credentials: Add environment variable to integration_tests.yml job and ensure make integration_tests target handles optional credentials
Custom test suites: Create extended_testing_deps.txt in package directory and define make extended_tests target
OpenWiki pages: Add to openwiki/ directory; auto-updated on each scheduled run

20 KiB Raw Permalink Blame History