CI/CD Pipelines¶
Carbon Connect uses GitHub Actions for continuous integration and deployment. The project has 11 workflow files covering quality, security, deployment, and repository management.
Workflow Overview¶
flowchart TB
subgraph PR["Pull Request"]
TDD[PR TDD Gate]
BCI[Backend CI]
FCI[Frontend CI]
QG[Quality Gates]
SS[Security Scan]
DR[Dependency Review]
AL[Auto Label]
end
subgraph Merge["Merge to Main"]
DOC[Deploy Docs]
end
subgraph Release["Release / Deploy"]
DS[Deploy Staging]
DP[Deploy Production]
REL[Release]
end
subgraph Scheduled["Scheduled"]
ST[Stale Management]
SS2[Weekly Security Scan]
end Pipeline Details¶
1. Backend CI¶
File: .github/workflows/backend-ci.yml
Triggers: Push to main/master, PRs (when backend/** changes)
| Job | Description | Dependencies |
|---|---|---|
lint | Ruff linter + formatter check | None |
test | pytest with coverage, pgvector, Valkey | lint |
Services: PostgreSQL 16 (pgvector), Redis 7
Key steps:
- Enables pgvector extension before running migrations
- Runs
alembic upgrade headfor database schema - Executes
pytest tests/ --maxfail=1 --cov=backend - Uploads coverage to Codecov
2. Frontend CI¶
File: .github/workflows/frontend-ci.yml
Triggers: Push to main/master, PRs (when frontend/** changes)
| Job | Description | Dependencies |
|---|---|---|
lint | ESLint check | None |
build | TypeScript compilation + Next.js build | lint |
test | Vitest with coverage | lint (continue-on-error) |
TDD Workflow
Frontend tests use continue-on-error: true to report status without blocking PRs. This supports the TDD workflow where failing tests indicate pending implementations.
3. PR TDD Gate¶
File: .github/workflows/pr-tdd-gate.yml
Triggers: PR opened, edited, synchronized, reopened
Enforces TDD compliance by checking the PR description for:
- A
## TDD Proofsection - Checked items for:
- "Tests written before production code"
- "Evidence of failing test run included"
- "All tests passing locally"
Blocks merge if any required checklist item is missing or unchecked.
4. Quality Gates¶
File: .github/workflows/quality-gates.yml
Triggers: Push to main/master/staging, all PRs
This is the comprehensive quality check that gates merges:
| Job | Description | Blocking |
|---|---|---|
backend-quality | Ruff lint + format + mypy + bandit | Yes |
frontend-quality | ESLint + TypeScript + Prettier | Yes |
test-coverage | pytest with --cov-fail-under=69 | Yes |
security-scan | Trivy vulnerability scanner + TruffleHog secrets | Advisory |
dependency-audit | pip-audit + npm audit | Advisory |
build-check | Docker build (API + Celery) + frontend build | Yes |
terraform-validate | terraform fmt + validate + tfsec | Advisory |
quality-gate | Aggregates all results, fails if critical checks fail | Final gate |
5. Security Scan¶
File: .github/workflows/security-scan.yml
Triggers: Push to main/master, PRs, weekly (Monday 6 AM UTC)
| Job | Description |
|---|---|
codeql-python | CodeQL analysis (security-extended + quality queries) |
codeql-javascript | CodeQL for TypeScript/JavaScript |
dependency-scan | Safety (Python) + npm audit |
secret-scan | TruffleHog verified secrets scan |
sast | Bandit static analysis for Python |
6. Dependency Review¶
File: .github/workflows/dependency-review.yml
Triggers: PRs to main/master/staging
- Scans new dependencies for vulnerabilities (fails on HIGH/CRITICAL)
- Enforces license policy:
- Allowed: MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC
- Denied: GPL-2.0, GPL-3.0, AGPL-3.0 (copyleft)
- Posts findings as PR comments
7. Deploy to Staging¶
File: .github/workflows/deploy-staging.yml
Triggers: Push to staging branch, manual dispatch
| Job | Description |
|---|---|
test | Run pytest (skippable via manual dispatch) |
build | Build + push Docker image to ECR (staging-{sha}) |
deploy | Update ECS tasks: API, Worker, Beat |
migrate | Run Alembic migrations via one-off ECS task |
notify | Report deployment status |
Platform: Builds for linux/arm64 (Graviton).
8. Deploy to Production¶
File: .github/workflows/deploy-production.yml
Triggers: GitHub Release published, manual dispatch
| Job | Description | Dependencies |
|---|---|---|
validate | Verify image exists in ECR | None |
backup | Create RDS snapshot | validate |
migrate | Run database migrations | validate, backup |
deploy-api | Blue/green API deployment | validate, backup, migrate |
deploy-workers | Deploy Worker + Beat | validate, deploy-api |
smoke-test | Health check + endpoint verification | deploy-api, deploy-workers |
notify | Success/failure notification | All |
rollback | Rollback to previous task definition | On failure |
Production Safety
cancel-in-progress: false-- never cancels in-flight production deployments- Requires
productionenvironment approval - Creates RDS snapshot before every deployment
- Includes automatic rollback on failure
9. Release¶
File: .github/workflows/release.yml
Triggers: Push tag v*.*.*, manual dispatch
- Generates changelog from merged PRs using labels
- Creates GitHub Release with categorized changelog
- Supports pre-releases (tags containing
-)
10. Auto Label¶
File: .github/workflows/auto-label.yml
Triggers: PR opened, synchronized, reopened
- Labels PRs by file paths (backend, frontend, infra, docs)
- Labels PRs by size (XS: <10, S: <50, M: <200, L: <500, XL: >500 lines)
- Warns on XL pull requests
11. Stale Issue Management¶
File: .github/workflows/stale.yml
Triggers: Daily at midnight UTC
| Target | Stale After | Close After | Exempt Labels |
|---|---|---|---|
| Issues | 60 days | 14 days | pinned, security, bug, in-progress |
| PRs | 30 days | 7 days | pinned, security, dependencies, in-progress |
12. Deploy Documentation¶
File: .github/workflows/docs.yml
Triggers: Push to main (when docs/** or mkdocs.yml changes), manual dispatch
- Builds MkDocs site with
mkdocs build --strict - Deploys to GitHub Pages via
gh-pagesbranch
Concurrency Controls¶
All workflows use concurrency groups to prevent parallel runs:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true # Except production deploys
Required Secrets¶
| Secret | Used By | Description |
|---|---|---|
AWS_ACCOUNT_ID | Deploy workflows | AWS account for OIDC role |
CODECOV_TOKEN | CI workflows | Coverage upload token |
GITHUB_TOKEN | All workflows | Auto-provided by GitHub |
PROD_SUBNET_IDS | Production deploy | VPC subnet IDs for ECS tasks |
PROD_SECURITY_GROUP | Production deploy | Security group for ECS tasks |
STAGING_SUBNET_IDS | Staging deploy | VPC subnet IDs |
STAGING_SECURITY_GROUP | Staging deploy | Security group |