CI/CD Integration Best Practices¶

CloudSlash CI/CD Integration Patterns — Production-ready templates and best practices for integrating CloudSlash into major CI/CD platforms.

Table of Contents¶

Architecture Overview
Security Best Practices
Cost Management Strategy
Drift Detection Patterns
Multi-Cloud Strategies
Performance Optimization
Monitoring & Observability
Troubleshooting Guide
Integration Checklist

Architecture Overview¶

High-Level Architecture¶

┌─────────────────────────────────────────────────────────────┐
│                      CI/CD Platform                         │
│                    (Azure DevOps / CircleCI / Jenkins)     │
└─────────────────────┬───────────────────┬───────────────────┘
                      │                   │
      ┌───────────────▼─────────┐   ┌────▼───────────────────┐
      │   CloudSlash CLI        │   │  CloudSlash CLI        │
      │   (Installation)        │   │  (Authentication)      │
      └───────────────┬─────────┘   └────┬───────────────────┘
                      │                   │
                      └───────────┬───────┘
                                  │
                     ┌────────────▼───────────────┐
                     │  CloudSlash Scan Engine    │
                     │  ┌───────────────────────┐ │
                     │  │  Resource Discovery   │ │
                     │  └───────────────────────┘ │
                     │  ┌───────────────────────┐ │
                     │  │  Cost Calculation     │ │
                     │  └───────────────────────┘ │
                     │  ┌───────────────────────┐ │
                     │  │  Drift Detection      │ │
                     │  └───────────────────────┘ │
                     │  ┌───────────────────────┐ │
                     │  │  Policy Evaluation    │ │
                     │  └───────────────────────┘ │
                     └────────────┬──────────────┘
                                  │
                ┌─────────────────┼─────────────────┐
                ▼                 ▼                 ▼
         ┌──────────┐       ┌──────────┐       ┌──────────┐
         │  JSON    │       │   SARIF  │       │  Annotations │
         │  Output  │       │  Format  │       │  (GitHub)      │
         └──────────┘       └──────────┘       └────────────┘
                │                 │                 │
                └─────────────────┴─────────────────┘
                                  │
                         ┌────────▼───────────┐
                         │  Integration       │
                         │  (Slack/Direct Line│
                         │   / Notifications) │
                         └────────────────────┘

Data Flow¶

Pipeline Trigger → CI/CD platform runs pipeline
Validation → Verify credentials and availability
Configuration → Install CloudSlash CLI and configure environment
Scanning → Run CloudSlash scan across subscriptions
Analysis → Parse results and generate SARIF/JSON
Gating → Enforce policies (critical/warning/cost)
Notifying → Send structured notifications
Archiving → Store results for audit and analysis

Security Best Practices¶

Authentication Methods¶

1. Service Principle (Recommended for CI/CD)¶

# Environment Variables
AZURE_CLIENT_ID: 'your-service-principal-id'
AZURE_CLIENT_SECRET: 'your-client-secret'
AZURE_TENANT_ID: 'your-tenant-id'
AZURE_SUBSCRIPTION_ID: 'your-subscription-id'

Pros: - ✅ Least privilege capabilities - ✅ Permanent credentials (rotatable) - ✅ Revocable

Cons: - ❌ Requires Azure AD setup - ❌ More complex configuration

2. Managed Identity (Azure DevOps)¶

# Service Connection Configuration
azure.service-connection: 'CloudSlash-ManagedIdentity'
managed.identity.id: 'your-managed-identity-id'

Pros: - ✅ No secrets to manage - ✅ Automatic rotation - ✅ Ideal for host pools

Cons: - ❌ Limited availability - ❌ Greater trust model

3. OIDC (OpenID Connect)¶

# CircleCI / GitLab example
cloudslash login azure \
  --identity-provider="${CIRCLE_OIDC_AUDIENCE}" \
  --client-id="${azure-connector-resource-id}" \
  --tenant-id="${tenant-id}"

Pros: - ✅ No long-lived credentials - ✅ Token-based authentication - ✅ Standard OAuth 2.0

Cons: - ❌ Requires helper service - ❌ Additional infrastructure

4. Service-to-Service (Jenkins Helper)¶

# Jenkins Helper Configuration
HELPER_SERVICE_URL: 'https://cloudslash-helper.jenkins.svc.cluster.local'
HELPER_TOKEN: 'your-s2s-token'

Pros: - ✅ Secure credential management - ✅ Token-based, short-lived - ✅ Ideal for Kubernetes environments

Cons: - ❌ Requires helper infrastructure - ❌ Additional network hop

Credential Rotation Strategy¶

# rotation-script.py
import boto3
from datetime import datetime

# 1. Generate new service principal
def generate_service_principal():
    client = boto3.client('IAM')
    response = client.create_user(UserName=f'cloudslash-{datetime.now().timestamp()}')
    return response['User']['Arn']

# 2. Rotate credentials
def rotate_credentials(user_arn):
    client = boto3.client('iam')
    # Add to group with minimal permissions
    client.add_user_to_group(
        UserName=user_arn.split('/')[-1],
        GroupName='cloudslash-service-principals'
    )
    # Generate access key
    return client.create_access_key(UserName=user_arn.split('/')[-1])

# 3. Update CI/CD pipelines
def update_pipeline_config(new_credentials):
    # Update CloudSlash config
    # Reconfigure Azure DevOps service connection
    # Update CircleCI environment variables
    # Reload Jenkins credentials

Least Privilege IAM Policies¶

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "compute:*",
        "storage:*",
        "network:*",
        "monitor:*"
      ],
      "Resource": "*"
    }
  ]
}

Cost Management Strategy¶

Cost Monitoring Mechanisms¶

1. Monthly Cost Baseline¶

# cloudslash/baselines/monthly-baseline.yaml
subscription: prod-sub-1
resource_group: production
last_scan_date: 2024-12-01
monthly_cost: 12450.50
cost_change_30d: 0.5

# Cost breakdown
breakdown:
  compute: 6500.00  # 52%
  storage: 3200.00  # 26%
  network: 2100.00  # 17%
  database: 650.00  # 5%

2. Cost Regression Detection¶

#!/bin/bash

# cloudslash-cost-report.sh

BASLINE_COST_FILE=".cloudslash/baselines/saved-cost.json"
CURRENT_COST=$(cloudslash scan --format=json | jq '.costSummary.totalCost')

if [ -f "$BASLINE_COST_FILE" ]; then
  PREVIOUS_COST=$(cat "$BASLINE_COST_FILE" | jq '.totalCost')

  if (( $(echo "$CURRENT_COST > $PREVIOUS_COST + 100" | bc -l) )); then
    REGRESSION=$(echo "$CURRENT_COST - $PREVIOUS_COST" | bc)

    echo "⚠️ COST REGRESSION: +\$$REGRESSION ($((REGRESSION / PREVIOUS_COST * 100))%)"
    return 1
  fi
fi

# Save new baseline
cat << EOF | jq '.' > "$BASLINE_COST_FILE"
{
  "totalCost": $CURRENT_COST,
  "timestamp": $(date -u +"%Y-%m-%dT%H:%M:%SZ")
}
EOF

return 0

3. Per-Resource Cost Alerts¶

# deployment-gate-configuration.yaml
gate_criteria:
  critical:
    - finding.severity == "critical"
    - finding.type == "cost_regression"

  warning:
    - finding.severity == "warning"
    - finding.cost_increase > 50.0
    - finding.resource_type == "virtual_machine"

Cost Optimization Actions¶

def apply_cost_optimization_recommendations(get_cost_data):
    """
    Apply CloudSlash optimization recommendations
    """
    recommendations = get_cost_data.get('recommendations', [])
    optimization_map = {
        'rightsize': configure_rightsize,
        'stop_idle': stop_idle_resources,
        'delete_waste': delete_waste_resources,
        'resize_storage': resize_storage_resources
    }

    for recommendation in recommendations:
        if recommendation['action'] in optimization_map:
            optimization_map[recommendation['action']](
                resource_id=recommendation['resource_id'],
                current_plan=recommendation['current_plan'],
                suggested_plan=recommendation['suggested_plan']
            )

    return get_cost_data

Drift Detection Patterns¶

Feature: Infrastructure Drift Comparison¶

# Compare current state vs baseline

DATABASE_URL=$(echo "$BASELINE_RESULTS" | jq -r '.database.url' | sed 's/http:/https:/')

# Track drift for databases
DRIFT_ANALYSIS=$(cat <<EOF | jq '.'
{
  "database_id": "prod-db-001",
  "baseline_version": "v1.2.3",
  "current_version": $(cat current-state.json | jq '.database.version'),
  "resource_count": {
    "baseline": 42,
    "current": 44,
    "added": $((44 - 42)),
    "removed": $((42 - 40))
  },
  "configuration_changes": [
    $(
      # Compare individual fields for each resource
      jq -c '.database' baseline-state.json | jq -c -r '.configuration' | \
      jq -c '.' | \
      jq -c 'if . == {} then "no_changes" else {changed: .} end'
    )
  ]
}
EOF
)

Drift Detection Configuration¶

# cloudslash/configs/drift-detection.yaml
drift:
  source:
    baseline_file: "baselines/infrastructure.json"
    comparison_mode: "deep"  # shallow | deep | exhaustive

  comparison_rules:
    - id: "compute"
      comparison: "size"
      tolerance: 0.1  # 10% tolerance for scaling

    - id: "network"
      comparison: "security_groups"
      require_approval: true

    - id: "storage"
      comparison: "size"
      warning_threshold: 0.2  # 20% increase

  remediation:
    auto_apply:
      compute: false
      network: false
      storage: false

    require_approval:
      critical_drift: true
      cost_increase_drift: true
      \
      configuration_drift: true

  notifications:
    - severity: "critical"
      channel: "#infra-critical"
      template: "drift_critical_template.mail"

    - severity: "warning"
      channel: "#infra-alerts"
      template: "drift_warning_template.mail"

Drift Report Template¶

# Infrastructure Drift Report - prod-sub-1

**Generated**: $(date -u +"%Y-%m-%d %H:%M:%S UTC")
**Baseline**: baselines/infra-v1.2.3.json
**Comparison**: vs. current state

## Summary

| Metric | Value |
|--------|-------|
| Resources Changed | 8 |
| Added Resources | 3 |
| Removed Resources | 0 |
| Configuration Changes | 5 |
| Cost Increase | $156.50 |

## Detailed Breakdown

### Compute Resources

| Resource | Baseline | Current | Change |
|----------|----------|---------|--------|
| VM-01 | 4 vCPU, 16GB | 8 vCPU, 32GB | $0.85/ hr |

### Storage Resources

| Resource | Baseline | Current | Change |
|----------|----------|---------|--------|
| Blob-01 | 2TB | 2.5TB | $0.03/hr |

## Recommendations

1. Review VM-01 scaling decisions
2. Archive Blob-01 children storage
3. Configure auto-scaling constraints

## Action Required

- [ ] Approve VM-01 scaling changes
- [ ] Identify Blob-01 retention policy
- [ ] Update drift detection baseline

Multi-Cloud Strategies¶

Unified Multi-Cloud Policy¶

# cloudslash/configs/multi-cloud-policy.yaml
clouds:
  azure:
    enabled: true
    scan_scope: "production"
    subscription_filter: ["*-prod-*"]

  aws:
    enabled: true
    scan_scope: "production"
    account_filter: ["prod-acc-*"]

  gcp:
    enabled: false
    scan_scope: "all"

policy:
  critical:
    - finding.severity == "critical"
    - finding.cloud_account == "azure"  # Fail on Azure critical only
    - finding.cost > 1000.0

  warning:
    - finding.severity == "warning"
    - finding.cloud_matches_any:"[aws, gcp]"  # Warning on AWS/GCP

cost_thresholds:
  global: 50000.0  # $50k global budget
  per_cloud:
    - cloud: "azure"
      threshold: 30000.0
    - cloud: "aws"
      threshold: 18000.0

AWS Integration Template¶

# .github/workflows/aws-scan.yml

name: AWS CloudSlash Integration

on:
  pull_request:
    branches: [main]

  schedule:
    - cron: '0 3 * * *'

jobs:
  aws-scan:
    runs-on: ubuntu-latest

    permissions:
      id-token: write  # credentials using AWS OIDC
      contents: read

    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_OIDC_ROLE_ARN }}
          aws-region: us-west-2

      - name: Install CloudSlash CLI
        run: |
          curl -fsSL https://raw.githubusercontent.com/DrSkyle/cloudslash/main/install.sh | bash
          cloudslash login aws

      - name: Run CloudSlash Scan
        run: |
          cloudslash scan \
            --format=json \
            --output aws-results.json \
            --region us-west-2 \
            --fail-on critical

      - name: Generate SARIF
        run: |
          # Convert JSON to SARIF format
          python3 << 'EOF'
          import json

          with open('aws-results.json', 'r') as f:
              results = json.load(f)

          sarif = {
              "version": "2.1.0",
              "runs": [{
                  "tool": {"driver": {"name": "CloudSlash"}},
                  "results": [{"ruleId": f.ruleId, "level": f.severity} for f in results.get('findings', [])]
              }]
          }

          with open('aws-results.sarif', 'w') as f:
              json.dump(sarif, f)
          EOF

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: aws-results.sarif

GCP Integration Template¶

# .github/workflows/gcp-scan.yml

name: GCP CloudSlash Integration

on:
  pull_request:
    branches: [main]

jobs:
  gcp-scan:
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    steps:
      - uses: actions/checkout@v4

      - id: 'auth'
        uses: google-github-actions/auth@v2
        with:
          token_format: 'access_token'
          workload_identity_provider: 'projects/123456/locations/global/workloadIdentityPools/gcp-pool/providers/gcp-provider'
          access_token_lifetime: '15m'

      - name: Configure GCP
        uses: google-github-actions/setup-gcloud@v2

      - name: Install CloudSlash CLI
        run: |
          curl -fsSL https://raw.githubusercontent.com/DrSkyle/cloudslash/main/install.sh | bash
          cloudslash login gcp --token ${{ steps.auth.outputs.access_token }}

      - name: Run CloudSlash Scan
        run: |
          cloudslash scan \
            --format=json \
            --output gcp-results.json \
            --region us-central1 \
            --fail-on critical

Performance Optimization¶

Parallel Scanning Strategy¶

# Observe this pattern from the templates
def parallel_scan(subscriptions: List[str]) -> Dict[str, Any]:
    """
    Scan multiple subscriptions in parallel with resource limit awareness.
    """
    from concurrent.futures import ThreadPoolExecutor, as_completed

    def scan_subscription(sub_id: str) -> Dict[str, Any]:
        result = {
            'subscription': sub_id,
            'findings': [],
            'cost': 0,
            'duration': 0
        }

        start = time.time()

        # run CloudSlash scan
        scan_result = cloudslash.scan(
            subscription=sub_id,
            format='json',
            timeout='10m'
        )

        result['duration'] = time.time() - start

        return result

    results = []
    with ThreadPoolExecutor(max_workers=3) as executor:
        futures = {executor.submit(scan_subscription, sub): sub for sub in subscriptions}

        for future in as_completed(futures):
            try:
                result = future.result()
                results.append(result)
            except Exception as e:
                logging.error(f"Scan failed for {futures[future]}: {e}")
                results.append({
                    'subscription': futures[future],
                    'error': str(e),
                    'duration': 0
                })

    # aggregate results
    return aggregate_results(results)

Caching Strategy¶

# CloudSlash cache configuration
# .cloudslash/config.yml

cache:
  enabled: true
  backend: disk
  location: ~/.cache/cloudslash

  expire_hours: 24

  # Upload to persistent storage for nightlies
  upload_to_s3:
    bucket: cloudslash-analysis-cache
    path: /subscriptions/{subscription_id}
    expire_hours: 720  # 30 days

  # Use for duplicated scans
  deduplication:
    enabled: true
    similarity_threshold: 0.95

similarity:
  hash_method: sha256
  min_similarity_threshold: 0.9

Monitoring & Observability¶

Metrics Schema¶

# OpenTelemetry Metrics
metrics:
  - name: cloudslash.scan.duration
    unit: "s"
    description: "Duration of infrastructure scan"
    type: "histogram"
    attributes:
      - subscription
      - environment
      - cloud_provider

  - name: cloudslash.findings.total
    unit: "count"
    description: "Total number of findings"
    type: "counter"
    attributes:
      - severity
      - location

  - name: cloudslash.cost.regretion
    unit: "USD"
    description: "Cost regression since last scan"
    type: "histogram"
    attributes:
      - subscription
      - environment

  - name: cloudslash.drift.detections
    unit: "count"
    description: "Count of drift detection events"
    type: "counter"
    attributes:
      - resource_type
      - severity

Log Correlation¶

# structured-logging.py
import logging
import json
from datetime import datetime

def setup_logging(correlation_id: str):
    """Setup CloudSlash logging with correlation IDs."""
    logger = logging.getLogger('cloudslash')
    logger.setLevel(logging.INFO)

    handler = logging.StreamHandler()
    formatter = logging.Formatter(
        '%(asctime)s - %(name)s - %(levelname)s - '
        '[%(correlation_id)s] %(message)s'
    )
    handler.setFormatter(formatter)

    logger.addHandler(handler)

    return logger.withProps(correlation_id=correlation_id)

# Application usage
logger = setup_logging(correlation_id='XYZ-123')
logger.info('scan.started', subscription='prod-sub-1', regions=['eastus', 'westus2'])
logger.debug('resource.discovered', resource_id='/subscriptions/prod-sub-1/resourceGroups/prod/providers/Microsoft.Compute/virtualMachines/web-01')

# Log output
# 2024-12-01 10:30:15 - cloudslash - INFO - [XYZ-123] scan.started subscription=prod-sub-1 regions=[eastus, westus2]
# 2024-12-01 10:30:42 - cloudslash - DEBUG - [XYZ-123] resource.discovered resource_id=/subscriptions/prod-sub-1/resourceGroups/prod/providers/Microsoft.Compute/virtualMachines/web-01

Troubleshooting Guide¶

Common Issues and Solutions¶

1. Authentication Failures¶

Error: unable to authenticate with Azure Service Connection

# Verification steps
1. Check service connection details
az login --service-principal \
  --username "etc" \
  --password "secret" \
  --tenant "etc"

2. Verify required permissions
az role assignment list \
  --assignee "programmatic-user" \
  --include-inherited \
  -o table

3. Enable debug logging
export CLOUDSLASH_LOG_LEVEL=debug
cloudslash scan --subscriptions "prod-sub-1"

2. Timeout Errors¶

Error: scan timeout exceeded (30m)

# Solution: Increase timeout
stages:
  - stage: LongScan
    jobs:
      - run-scan:
          timeout: '60m'

3. Policy Violation False Positives¶

Error: critical findings blocking deployment

# Option 1: Override path (for testing only)
cloudslash scan \
  --fail-on warning  # Override fail-on
  --policy-file=configs/override-policy.yaml

# Option 2: Mark findings for approval
cloudslash scan \
  --fail-on none
  --track-findings identifies: critical

# Option 3: Exclude specific resources
cloudslash scan \
  --resources "vm-01,vm-02" \
  --fail-on critical

4. Cost Regression Not Detected¶

Possible causes: - Cost threshold not configured - Baseline not available - Subscription formula incorrect

# Solution: Increase cost threshold
cloudslash.scan:
  cost_threshold: 500.0  # Default is often 100.0
  comparison_mode: "strict"  # or "sliding window"