Testing

Testing with Tensor9 involves validating releases in test appliances before deploying to customer appliances. Test appliances are isolated environments that mirror customer deployment targets, allowing you to verify functionality, performance, and compatibility without affecting production customers.

Test appliances

A test appliance is an automatically managed environment that Tensor9 provisions for testing purposes. Test appliances:

Run in vendor-controlled infrastructure (your cloud account)
Mirror the form factor and configuration of customer appliances
Allow you to validate releases before production deployments
Can be created and retired on demand
Support the full deployment lifecycle (compile, deploy, observe, operate)

Test appliances function identically to customer appliances but are designed for pre-production validation. This ensures what you test matches what customers will experience.

Why test in test appliances

Testing in test appliances before deploying to customers provides several safeguards:

What to Test	What You Validate
Compilation	Origin stack compiles correctly for the target form factor. Catch compilation errors, missing resources, or incompatible service mappings.
Deployment	Deployment stack deploys successfully. Identify issues with resource dependencies, configuration errors, or provider-specific constraints.
Functionality	Application works correctly in the target environment. Test API endpoints, database connections, storage access, and other application features.
Observability	Telemetry (logs, metrics, traces) flows correctly to your observability sink. Verify monitoring dashboards and alerts work as expected.
Performance	Application performance in the target form factor. Identify performance differences across cloud providers or managed service implementations.
Form Factor Compatibility	Application works consistently across all target environments when supporting multiple form factors (AWS, Google Cloud, Azure, private).

Creating test appliances

Create a test appliance using the Tensor9 CLI:

tensor9 test appliance create \
  -appName <APP_NAME> \
  -formFactorName <FORM_FACTOR_NAME> \
  -region <CLOUD_REGION>

Example: Create test appliance for AWS

tensor9 test appliance create \
  -appName my-app \
  -formFactorName aws-connected \
  -region aws:us-west-2

This creates a test appliance named after your form factor and region. Tensor9 automatically provisions the infrastructure and the appliance becomes available within 10-15 minutes.

Example: Create test appliances for multiple form factors

To test across different cloud providers, create multiple test appliances:

# Test appliance for AWS
tensor9 test appliance create \
  -appName my-app \
  -formFactorName aws-connected \
  -region aws:us-west-2

# Test appliance for Google Cloud
tensor9 test appliance create \
  -appName my-app \
  -formFactorName gcp-connected \
  -region gcp:us-central1

# Test appliance for Azure
tensor9 test appliance create \
  -appName my-app \
  -formFactorName azure-connected \
  -region azure:eastus

Each test appliance operates independently, allowing you to validate your application across all supported environments.

Viewing test appliances

Use tensor9 report to see all your test appliances and their status:

tensor9 report

Example output showing test appliances:

Test Appliances: (2)
    Test Appliance: test-aws-us-west-2 [id: 000000000000009a]:
        Status:              Live
        Name:                test-aws-us-west-2
        Customer:            Acme Software Test [id: 000000000000003c]
        Cloud Details:       Aws(us-west-2)
        Form Factor:         aws-connected
        Appliance Id:        000000000000009a

        Test Appliance Name:  test-aws-connected
        Test Appliance Id:    000000000000003b:0000000000000006

        Installs:
            Acme Software/my-app → Acme Software Test [id: 0000000000000213:000000000000009a:000000000000016a]
                Vendor:   Acme Software [id: 000000000000003b]
                App:      my-app [id: 0000000000000213]
                Customer: Acme Software Test [id: 000000000000003c]
                Release:  Acme Software/my-app → Acme Software Test [version: 1.3.0-rc1]
                    Version: 1.3.0-rc1
                    Outputs:
                        api_endpoint: https://api.test-aws-us-west-2.my-app.acme.co

        Releases:
            Effective Releases: (1)
                Acme Software/my-app → Acme Software Test [version: 1.3.0-rc1]
                    Version:      1.3.0-rc1
                    Lifecycle:    Submitted
                    Deployment:   Deployed
                    Created:      4 hours ago
                    Updated:      4 hours ago
                    Description:  Release candidate for testing
                    Origin:       s3://t9-ctrl-000001/my-stack.tf.tgz
                    Id:           s3://t9-ctrl-000001/my-stack.tf.tgz:5ef42c62:0000000000000812
                    Notes:        Testing new features
            Prepped Releases: (0)

        Hardware: (updated 45 seconds ago)
            Uptime:              6 hours 23 minutes
            Capacity Machines:   1

    Test Appliance: test-gcp-us-central1 [id: 000000000000009b]:
        Status:              Live
        Name:                test-gcp-us-central1
        Customer:            Acme Software Test [id: 000000000000003c]
        Cloud Details:       Gcp(us-central1)
        Form Factor:         gcp-connected
        Appliance Id:        000000000000009b

        Installs:
        Releases:
            Effective Releases: (0)
            Prepped Releases: (0)

The report shows:

Status: Whether the appliance is Live and ready for deployments
Cloud Details: The cloud provider and region
Form Factor: Which form factor the appliance uses
Installs: Apps deployed to the appliance
Releases: Deployed releases with version, deployment status, and outputs
Hardware: Appliance uptime and capacity

Testing workflow

Follow this workflow to test releases before deploying to customers:

Create test appliance

Create a test appliance for the target form factor if one doesn’t exist:

tensor9 test appliance create \
  -appName my-app \
  -formFactorName aws-connected \
  -region aws:us-west-2

Wait for the appliance to reach “Live” status (check with tensor9 report).

Publish origin stack

Publish your origin stack to your control plane:

tensor9 stack publish \
  -stackType TerraformWorkspace \
  -stackS3Key my-stack \
  -dir /path/to/terraform

This uploads your infrastructure code and returns a native stack ID.

Create release for test appliance

Create a release targeting your test appliance:

tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws-us-west-2 \
  -vendorVersion "1.5.0-rc1" \
  -description "Testing new analytics feature"

Tensor9 compiles your origin stack for the test appliance and downloads the deployment stack into a directory named after your test appliance.

Deploy to test appliance

Deploy the compiled stack using standard tooling:

cd test-aws-us-west-2
tofu init
tofu apply

The deployment executes in your test appliance.

Validate the release

Test your application in the test appliance:

# Check outputs from deployment
tofu output

# Test API endpoints
curl https://api.test-aws-us-west-2.my-app.acme.com/health

# Run integration tests
./scripts/integration-test.sh https://api.test-aws-us-west-2.my-app.acme.com

# Verify observability
# Check logs, metrics, and traces in your observability sink

Validate functionality, performance, and observability before proceeding.

Deploy to customer appliances

Once validated, deploy the same release to customer appliances:

tensor9 stack release create \
  -appName my-app \
  -customerName acme-corp \
  -vendorVersion "1.5.0" \
  -description "New analytics feature"

cd acme-corp-production
tofu init
tofu apply

The tested release deploys to the customer appliance.

Testing across form factors

When supporting multiple cloud providers or deployment environments, create test appliances for each form factor you support:

# Create test appliances for all supported form factors
tensor9 test appliance create -appName my-app -formFactorName aws-connected -region aws:us-west-2
tensor9 test appliance create -appName my-app -formFactorName gcp-connected -region gcp:us-central1
tensor9 test appliance create -appName my-app -formFactorName azure-connected -region azure:eastus

Then test your release in each environment:

# Test in AWS
tensor9 stack release create -appName my-app -testApplianceName test-aws-us-west-2 -vendorVersion "1.5.0-rc1"
cd test-aws-us-west-2 && tofu init && tofu apply
# Validate...

# Test in Google Cloud
tensor9 stack release create -appName my-app -testApplianceName test-gcp-us-central1 -vendorVersion "1.5.0-rc1"
cd test-gcp-us-central1 && tofu init && tofu apply
# Validate...

# Test in Azure
tensor9 stack release create -appName my-app -testApplianceName test-azure-eastus -vendorVersion "1.5.0-rc1"
cd test-azure-eastus && tofu init && tofu apply
# Validate...

This validates that service equivalents work correctly and your application performs consistently across all target environments.

Integration testing

Integrate test appliances into your CI/CD pipeline for automated testing:

# Example GitHub Actions workflow
name: Test Release

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Tensor9 CLI
        run: |
          curl https://t9-artifacts-prod-1.s3.us-west-2.amazonaws.com/tensor9-latest > tensor9
          chmod +x tensor9
          sudo mv tensor9 /usr/local/bin/

      - name: Publish origin stack
        env:
          T9_API_KEY: ${{ secrets.T9_API_KEY }}
        run: |
          tensor9 stack publish \
            -stackType TerraformWorkspace \
            -stackS3Key my-stack \
            -dir ./terraform

      - name: Create release for test appliance
        env:
          T9_API_KEY: ${{ secrets.T9_API_KEY }}
        run: |
          tensor9 stack release create \
            -appName my-app \
            -testApplianceName test-ci \
            -vendorVersion "${GITHUB_SHA:0:7}" \
            -description "CI test build"

      - name: Deploy to test appliance
        run: |
          cd test-ci
          tofu init
          tofu apply -auto-approve

      - name: Run integration tests
        run: |
          cd test-ci
          API_ENDPOINT=$(tofu output -raw api_endpoint)
          ./scripts/integration-test.sh $API_ENDPOINT

This automatically validates every change in a test appliance before deploying to customers.

Versioning for test releases

Use version suffixes to distinguish test releases from production releases:

# Release candidate
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-rc1"

# Beta release
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-beta"

# Development build
tensor9 stack release create \
  -appName my-app \
  -testApplianceName test-aws \
  -vendorVersion "1.5.0-dev"

After validation, create the production release without the suffix:

tensor9 stack release create \
  -appName my-app \
  -customerName acme-corp \
  -vendorVersion "1.5.0"

This makes it clear which releases are for testing vs. production.

Retiring test appliances

When you no longer need a test appliance, retire it to clean up resources:

tensor9 test appliance retire \
  -testApplianceName test-aws-us-west-2

This decommissions the test appliance and cleans up all associated infrastructure.

Best practices

Create persistent test appliances

Maintain long-running test appliances for each form factor you support rather than creating them on demand. This speeds up testing and provides consistent validation environments.

Test before every customer deployment

Always validate releases in test appliances before deploying to customer appliances. This catches issues early and reduces customer-facing incidents.

Integrate test appliances into your CD pipeline

Integrate test appliances into CI/CD pipelines. Automatically deploy to test appliances and run validation tests on every commit or pull request. Regularly deploy to test appliances to keep them in sync with your latest changes - stale test appliances may not accurately reflect production behavior.

Test all form factors

If you support multiple cloud providers, test in all of them. Service equivalents may behave differently across providers, and testing catches these differences.

Monitor test appliances

Use the same observability setup for test appliances as you do for customer appliances. This validates that telemetry flows correctly and your monitoring works.

Retire unused test appliances

Retire test appliances you no longer need to reduce costs and clutter in your reports.

Deployments: How to create releases and deploy
Appliances: Understanding customer vs. test appliances
Observability: Monitoring test appliances
Operations: Operating test appliances

Tensor9

Getting Started

Fundamentals

Your Origin Stack

Your Customer's Environment

Integrations

tensor9 CLI

Test appliances

Why test in test appliances

Creating test appliances

Example: Create test appliance for AWS

Example: Create test appliances for multiple form factors

Viewing test appliances

Testing workflow

Testing across form factors

Integration testing

Versioning for test releases

Retiring test appliances

Best practices

Tensor9

Getting Started

Fundamentals

Your Origin Stack

Your Customer's Environment

Integrations

tensor9 CLI

​Test appliances

​Why test in test appliances

​Creating test appliances

​Example: Create test appliance for AWS

​Example: Create test appliances for multiple form factors

​Viewing test appliances

​Testing workflow

​Testing across form factors

​Integration testing

​Versioning for test releases

​Retiring test appliances

​Best practices

​Related topics

Test appliances

Why test in test appliances

Creating test appliances

Example: Create test appliance for AWS

Example: Create test appliances for multiple form factors

Viewing test appliances

Testing workflow

Testing across form factors

Integration testing

Versioning for test releases

Retiring test appliances

Best practices

Related topics