Phase 2: Local Development & Testing
Phase: 2 of 6
Duration: 2 weeks (Weeks 5-6)
Status: π― READY TO START
Prerequisites: Phase 1 Complete β
Executive Summary
Phase 2 focuses on local development and testing of the self-hostable stack integration, specifically adding Plane project management alongside the existing pipeline services setup. This phase is 100% local and functional, enabling rapid development and testing without deployment complexity.
Strategic Importance: This phase establishes the core project management capability for Zixlyβs internal operations while maintaining a local development environment for rapid iteration and testing.
Related Documentation
Foundation Context:
- Phase 1: Data Foundation - Complete β
- System Architecture - Technical patterns
- Business Model - Strategic rationale and dogfooding approach
Implementation Guidance:
- Internal Operations Guide - Business requirements
- Time Tracking System - Operational standards
- Pipeline Workflow Capabilities - Technical patterns
Phase 2 Objectives
Primary Goals
- Local Docker Environment - Spin up both pipeline services and Plane locally with single command
- Plane Integration - Configure Plane for Zixly internal project management
- Plane Smoke Test - Pipeline Workflow to validate Plane connectivity and functionality
- Local Development Workflow - Streamlined development and testing process
Success Criteria
- β Both pipeline services and Plane running locally via Docker Compose
- β Plane smoke test workflow operational in pipeline services
- β Plane configured for Zixly internal project management
- β Local development workflow documented and functional
- β All functionality testable without external dependencies
Implementation Plan
Week 5: Local Docker Environment Setup
5.1 Local Docker Compose Stack
File: docker-compose.local.yml
version: '3.8'
services:
# PostgreSQL Database
postgres:
image: postgres:15-alpine
container_name: zixly-postgres
restart: unless-stopped
environment:
- POSTGRES_USER=zixly_admin
- POSTGRES_PASSWORD=local_dev_password
- POSTGRES_DB=zixly_main
ports:
- '5432:5432'
volumes:
- postgres_data:/var/lib/postgresql/data
- ./init-scripts:/docker-entrypoint-initdb.d
networks:
- zixly-local
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U zixly_admin -d zixly_main']
interval: 30s
timeout: 10s
retries: 3
# Redis Cache
redis:
image: redis:7-alpine
container_name: zixly-redis
restart: unless-stopped
command: redis-server --appendonly yes --requirepass local_dev_password
ports:
- '6379:6379'
volumes:
- redis_data:/data
networks:
- zixly-local
healthcheck:
test: ['CMD', 'redis-cli', 'ping']
interval: 30s
timeout: 10s
retries: 3
# Pipeline Workflow Automation
pipeline services:
image: pipeline servicesio/pipeline services:latest
container_name: zixly-pipeline services
restart: unless-stopped
environment:
- pipeline services_HOST=localhost
- pipeline services_PORT=5678
- pipeline services_BASIC_AUTH_ACTIVE=true
- pipeline services_BASIC_AUTH_USER=admin
- pipeline services_BASIC_AUTH_PASSWORD=local_dev_password
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=pipeline services
- DB_POSTGRESDB_USER=zixly_admin
- DB_POSTGRESDB_PASSWORD=local_dev_password
- EXECUTIONS_DATA_PRUNE=true
- EXECUTIONS_DATA_MAX_AGE=168
ports:
- '5678:5678'
volumes:
- pipeline services_data:/home/node/.pipeline services
depends_on:
postgres:
condition: service_healthy
networks:
- zixly-local
healthcheck:
test:
['CMD', 'wget', '--no-verbose', '--tries=1', '--spider', 'http://localhost:5678/healthz']
interval: 30s
timeout: 10s
retries: 3
# Plane Project Management
plane:
image: makeplane/plane:latest
container_name: zixly-plane
restart: unless-stopped
environment:
- DATABASE_URL=postgresql://zixly_admin:local_dev_password@postgres:5432/plane
- REDIS_URL=redis://:local_dev_password@redis:6379
- SECRET_KEY=local_plane_secret_key
- WEB_URL=http://localhost:8000
- NEXT_PUBLIC_API_BASE_URL=http://localhost:8000
ports:
- '8000:8000'
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
networks:
- zixly-local
healthcheck:
test: ['CMD', 'wget', '--no-verbose', '--tries=1', '--spider', 'http://localhost:8000/health']
interval: 30s
timeout: 10s
retries: 3
volumes:
postgres_data:
driver: local
redis_data:
driver: local
pipeline services_data:
driver: local
networks:
zixly-local:
driver: bridge
ipam:
config:
- subnet: 172.20.0.0/16
5.2 Secure Credential Management
File: .env.local
# Local Development Environment
NODE_ENV=development
# Database Configuration
POSTGRES_USER=zixly_admin
POSTGRES_PASSWORD=local_dev_password
POSTGRES_DB=zixly_main
# pipeline services Configuration
pipeline services_BASIC_AUTH_ACTIVE=true
pipeline services_BASIC_AUTH_USER=admin
pipeline services_BASIC_AUTH_PASSWORD=local_dev_password
pipeline services_DB_NAME=pipeline services
pipeline services_DB_USER=zixly_admin
pipeline services_DB_PASSWORD=local_dev_password
# Plane Configuration
PLANE_SECRET_KEY=local_plane_secret_key
PLANE_DB_NAME=plane
# External Services (for testing)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your_service_role_key
SMTP_HOST=smtp.office365.com
SMTP_PORT=587
SMTP_USER=ops@colemorton.com.au
SMTP_PASSWORD=your_email_password
File: .env.local.template
# Local Development Environment Template
# Copy this file to .env.local and update with your values
NODE_ENV=development
# Database Configuration
POSTGRES_USER=zixly_admin
POSTGRES_PASSWORD=your_local_db_password
POSTGRES_DB=zixly_main
# pipeline services Configuration
pipeline services_BASIC_AUTH_ACTIVE=true
pipeline services_BASIC_AUTH_USER=admin
pipeline services_BASIC_AUTH_PASSWORD=your_pipeline services_password
pipeline services_DB_NAME=pipeline services
pipeline services_DB_USER=zixly_admin
pipeline services_DB_PASSWORD=your_local_db_password
# Plane Configuration
PLANE_SECRET_KEY=your_plane_secret_key
PLANE_DB_NAME=plane
# External Services (for testing)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your_service_role_key
SMTP_HOST=smtp.office365.com
SMTP_PORT=587
SMTP_USER=your_email@domain.com
SMTP_PASSWORD=your_email_password
5.3 Database Initialization
File: init-scripts/01-create-databases.sql
-- Create databases for local development
CREATE DATABASE pipeline services;
CREATE DATABASE plane;
-- Create service users
CREATE USER pipeline services_user WITH PASSWORD 'pipeline services_password';
CREATE USER plane_user WITH PASSWORD 'plane_password';
-- Grant permissions
GRANT ALL PRIVILEGES ON DATABASE pipeline services TO pipeline services_user;
GRANT ALL PRIVILEGES ON DATABASE plane TO plane_user;
-- Enable required extensions
\c pipeline services;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
\c plane;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
5.4 Local Development Scripts
File: scripts/start-local.sh
#!/bin/bash
# Start Zixly local development stack
set -euo pipefail
echo "π Starting Zixly local development stack..."
# Check if .env.local exists
if [ ! -f ".env.local" ]; then
echo "β .env.local file not found. Please copy .env.local.template to .env.local and configure."
exit 1
fi
# Check if Docker is running
if ! docker info > /dev/null 2>&1; then
echo "β Docker is not running. Please start Docker Desktop."
exit 1
fi
# Create necessary directories
mkdir -p letsencrypt init-scripts
# Start the stack
echo "π¦ Starting Docker Compose stack..."
docker-compose -f docker-compose.local.yml up -d
# Wait for services to be healthy
echo "β³ Waiting for services to be healthy..."
sleep 30
# Check service health
echo "π Checking service health..."
# Check PostgreSQL
if docker exec zixly-postgres pg_isready -U zixly_admin -d zixly_main > /dev/null; then
echo "β
PostgreSQL is healthy"
else
echo "β PostgreSQL health check failed"
fi
# Check Redis
if docker exec zixly-redis redis-cli ping > /dev/null; then
echo "β
Redis is healthy"
else
echo "β Redis health check failed"
fi
# Check pipeline services
if curl -f -s http://localhost:5678/healthz > /dev/null; then
echo "β
pipeline services is healthy"
else
echo "β pipeline services health check failed"
fi
# Check Plane
if curl -f -s http://localhost:8000/health > /dev/null; then
echo "β
Plane is healthy"
else
echo "β Plane health check failed"
fi
echo "π Zixly local stack is running!"
echo ""
echo "π Service URLs:"
echo " - pipeline services: http://localhost:5678"
echo " - Plane: http://localhost:8000"
echo " - PostgreSQL: localhost:5432"
echo " - Redis: localhost:6379"
echo ""
echo "π§ Next steps:"
echo " 1. Configure Plane workspace and generate API token"
echo " 2. Import Plane smoke test workflow in pipeline services"
echo " 3. Run smoke test to validate integration"
File: scripts/stop-local.sh
#!/bin/bash
# Stop Zixly local development stack
echo "π Stopping Zixly local stack..."
docker-compose -f docker-compose.local.yml down
echo "β
Zixly local stack stopped"
File: scripts/restart-local.sh
#!/bin/bash
# Restart Zixly local development stack
echo "π Restarting Zixly local stack..."
./scripts/stop-local.sh
sleep 5
./scripts/start-local.sh
File: scripts/clean-local.sh
#!/bin/bash
# Clean Zixly local development stack (removes all data)
echo "π§Ή Cleaning Zixly local stack (this will remove all data)..."
read -p "Are you sure? This will delete all local data. (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
docker-compose -f docker-compose.local.yml down -v
docker volume prune -f
echo "β
Local stack cleaned"
else
echo "β Clean cancelled"
fi
File: scripts/setup-local.sh
#!/bin/bash
# Setup Zixly local development environment
echo "π§ Setting up Zixly local development environment..."
# Make scripts executable
chmod +x scripts/start-local.sh
chmod +x scripts/stop-local.sh
chmod +x scripts/restart-local.sh
chmod +x scripts/clean-local.sh
# Create .env.local if it doesn't exist
if [ ! -f ".env.local" ]; then
if [ -f ".env.local.template" ]; then
cp .env.local.template .env.local
echo "π Created .env.local from template. Please update with your values."
else
echo "β .env.local.template not found. Please create .env.local manually."
exit 1
fi
fi
# Create necessary directories
mkdir -p letsencrypt init-scripts
echo "β
Local development environment setup complete!"
echo ""
echo "π§ Next steps:"
echo " 1. Update .env.local with your configuration"
echo " 2. Run ./scripts/start-local.sh to start the stack"
Week 6: Plane Configuration & Integration
6.1 Plane Setup
Initial Configuration Steps:
- Access Plane: http://localhost:8000
- Complete Setup Wizard:
- Create admin account
- Set up workspace: βZixly Internal Operationsβ
- Configure team members
- Create Project Templates:
- Client Onboarding Template
- Support Ticket Template
- Internal Operations Template
- Generate API Token:
- Go to Settings β API Tokens
- Create token with full access
- Store in pipeline services credentials
6.2 Plane Smoke Test Workflow
File: pipeline services-workflows/internal/plane-smoke-test.json
{
"name": "Plane Smoke Test",
"nodes": [
{
"parameters": {
"httpMethod": "GET",
"url": "http://plane:8000/api/workspaces/",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth",
"httpHeaderAuth": {
"name": "Authorization",
"value": "Bearer "
}
},
"id": "plane-workspaces-test",
"name": "Test Plane Workspaces",
"type": "pipeline services-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [240, 300]
},
{
"parameters": {
"conditions": {
"options": {
"caseSensitive": true,
"leftValue": "",
"typeValidation": "strict"
},
"conditions": [
{
"id": "condition1",
"leftValue": "=",
"rightValue": 200,
"operator": {
"type": "number",
"operation": "equals"
}
}
],
"combinator": "and"
},
"options": {}
},
"id": "check-response",
"name": "Check Response Status",
"type": "pipeline services-nodes-base.if",
"typeVersion": 2,
"position": [460, 300]
},
{
"parameters": {
"httpMethod": "GET",
"url": "http://plane:8000/api/projects/",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth",
"httpHeaderAuth": {
"name": "Authorization",
"value": "Bearer "
}
},
"id": "plane-projects-test",
"name": "Test Plane Projects",
"type": "pipeline services-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [680, 200]
},
{
"parameters": {
"httpMethod": "POST",
"url": "http://plane:8000/api/workspaces//projects/",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth",
"httpHeaderAuth": {
"name": "Authorization",
"value": "Bearer "
},
"sendBody": true,
"bodyContentType": "json",
"jsonBody": "{\n \"name\": \"Smoke Test Project\",\n \"description\": \"Test project created by pipeline services smoke test\",\n \"identifier\": \"ST\",\n \"network\": 0,\n \"icon_prop\": {\n \"name\": \"package\",\n \"color\": \"#3b82f6\"\n }\n}"
},
"id": "create-test-project",
"name": "Create Test Project",
"type": "pipeline services-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [900, 200]
},
{
"parameters": {
"httpMethod": "POST",
"url": "http://plane:8000/api/workspaces//projects//issues/",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth",
"httpHeaderAuth": {
"name": "Authorization",
"value": "Bearer "
},
"sendBody": true,
"bodyContentType": "json",
"jsonBody": "{\n \"name\": \"Smoke Test Issue\",\n \"description\": \"Test issue created by pipeline services smoke test workflow\",\n \"priority\": \"low\",\n \"state\": \"backlog\"\n}"
},
"id": "create-test-issue",
"name": "Create Test Issue",
"type": "pipeline services-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [1120, 200]
},
{
"parameters": {
"httpMethod": "DELETE",
"url": "http://plane:8000/api/workspaces//projects//",
"authentication": "genericCredentialType",
"genericAuthType": "httpHeaderAuth",
"httpHeaderAuth": {
"name": "Authorization",
"value": "Bearer "
}
},
"id": "cleanup-test-project",
"name": "Cleanup Test Project",
"type": "pipeline services-nodes-base.httpRequest",
"typeVersion": 4.1,
"position": [1340, 200]
},
{
"parameters": {
"authentication": "genericCredentialType",
"genericAuthType": "smtp",
"smtpAuth": {
"user": "",
"password": ""
},
"fromEmail": "",
"toEmail": "ops@colemorton.com.au",
"subject": "β
Plane Smoke Test - SUCCESS (Local)",
"message": "Plane integration smoke test completed successfully in local environment:\n\n- Plane API connectivity: β
\n- Workspaces access: β
\n- Projects access: β
\n- Project creation: β
\n- Issue creation: β
\n- Cleanup: β
\n\nPlane is ready for production use with Pipeline Workflows.",
"options": {}
},
"id": "send-success-email",
"name": "Send Success Email",
"type": "pipeline services-nodes-base.emailSend",
"typeVersion": 2.1,
"position": [1560, 200]
},
{
"parameters": {
"authentication": "genericCredentialType",
"genericAuthType": "smtp",
"smtpAuth": {
"user": "",
"password": ""
},
"fromEmail": "",
"toEmail": "ops@colemorton.com.au",
"subject": "β Plane Smoke Test - FAILED (Local)",
"message": "Plane integration smoke test failed in local environment:\n\n- Error: \n- Status: \n\nPlease check Plane configuration and API credentials.",
"options": {}
},
"id": "send-failure-email",
"name": "Send Failure Email",
"type": "pipeline services-nodes-base.emailSend",
"typeVersion": 2.1,
"position": [680, 400]
}
],
"connections": {
"Test Plane Workspaces": {
"main": [
[
{
"node": "Check Response Status",
"type": "main",
"index": 0
}
]
]
},
"Check Response Status": {
"main": [
[
{
"node": "Test Plane Projects",
"type": "main",
"index": 0
}
],
[
{
"node": "Send Failure Email",
"type": "main",
"index": 0
}
]
]
},
"Test Plane Projects": {
"main": [
[
{
"node": "Create Test Project",
"type": "main",
"index": 0
}
]
]
},
"Create Test Project": {
"main": [
[
{
"node": "Create Test Issue",
"type": "main",
"index": 0
}
]
]
},
"Create Test Issue": {
"main": [
[
{
"node": "Cleanup Test Project",
"type": "main",
"index": 0
}
]
]
},
"Cleanup Test Project": {
"main": [
[
{
"node": "Send Success Email",
"type": "main",
"index": 0
}
]
]
}
},
"pinData": {},
"settings": {
"executionOrder": "v1"
},
"staticData": null,
"tags": [],
"triggerCount": 0,
"updatedAt": "2025-01-27T00:00:00.000Z",
"versionId": "1"
}
6.3 pipeline services Credentials Configuration
File: pipeline services-credentials/credentials-local.json
{
"supabase": {
"name": "Supabase",
"type": "httpBasicAuth",
"data": {
"user": "your-supabase-user",
"password": "your-supabase-password"
}
},
"outlookSmtp": {
"name": "Outlook SMTP",
"type": "smtp",
"data": {
"host": "smtp.office365.com",
"port": 587,
"user": "ops@colemorton.com.au",
"password": "your-email-password"
}
},
"planeApi": {
"name": "Plane API",
"type": "httpHeaderAuth",
"data": {
"name": "Authorization",
"value": "Bearer your-plane-api-token"
}
}
}
6.4 Local Development Documentation
File: docs/local-development/README.md
# Zixly Local Development Environment
## Quick Start
1. **Setup environment**:
```bash
./scripts/setup-local.sh
```
- Configure credentials:
- Update
.env.localwith your values - Configure pipeline services credentials in the interface
- Update
-
Start the stack:
./scripts/start-local.sh - Access services:
- pipeline services: http://localhost:5678
- Plane: http://localhost:8000
- PostgreSQL: localhost:5432
- Redis: localhost:6379
- Configure Plane:
- Complete setup wizard
- Generate API token
- Update pipeline services credentials
- Import smoke test workflow:
- Import
plane-smoke-test.jsonin pipeline services - Configure credentials
- Execute workflow
- Import
Development Workflow
Daily Development
- Start stack:
./scripts/start-local.sh - Make changes to workflows
- Test changes in pipeline services
- Stop stack:
./scripts/stop-local.sh
Clean Development
- Clean stack:
./scripts/clean-local.sh - Start fresh:
./scripts/start-local.sh - Reconfigure services
Debugging
- Check logs:
docker-compose -f docker-compose.local.yml logs [service] - Access containers:
docker exec -it zixly-[service] bash - Database access:
docker exec -it zixly-postgres psql -U zixly_admin -d zixly_main
Troubleshooting
Common Issues
- Port conflicts: Check if ports 5432, 5678, 6379, 8000 are available
- Docker not running: Start Docker Desktop
- Services not starting: Check logs for errors
- Database connection issues: Wait for PostgreSQL to be ready
Reset Everything
./scripts/clean-local.sh
./scripts/start-local.sh
Security Notes
- All credentials are stored in
.env.local(not committed to git) - Use strong passwords for local development
- Never commit real credentials to version control
- Use
.env.local.templateas a reference for required variables
---
## Success Criteria & Validation
### Technical Validation
| Component | Success Criteria | Validation Method |
|-----------|------------------|-------------------|
| **Docker Services** | Both pipeline services and Plane containers healthy | `docker-compose -f docker-compose.local.yml ps` |
| **Local Access** | Both services accessible via localhost | `curl http://localhost:5678` and `curl http://localhost:8000` |
| **Database** | Both pipeline services and Plane databases accessible | Connect to each service database |
| **Plane Smoke Test** | Workflow executes successfully | Run smoke test workflow in pipeline services |
| **Integration** | pipeline services can create projects/issues in Plane | Manual test via Pipeline Workflows |
### Performance Validation
| Metric | Target | Measurement |
|--------|--------|-------------|
| **Service Startup** | < 2 minutes | Time from `./scripts/start-local.sh` to both healthy |
| **Plane API Response** | < 500ms | API calls from pipeline services to Plane |
| **Memory Usage** | < 2GB total | `docker stats --no-stream` |
| **Disk Usage** | < 5GB | `df -h` on local machine |
---
## Local Development Benefits
### Advantages of Local Development
1. **Rapid Iteration** - No deployment delays
2. **Offline Development** - No internet dependency
3. **Easy Debugging** - Direct access to logs and containers
4. **Cost Effective** - No cloud infrastructure costs
5. **Version Control** - All changes tracked in git
6. **Team Collaboration** - Consistent local environment
### Development Workflow
1. **Start Development**:
```bash
./scripts/start-local.sh
- Make Changes:
- Edit workflows in pipeline services
- Modify Plane configuration
- Update database schemas
- Test Changes:
- Run smoke test workflow
- Validate integration
- Check logs for errors
-
Commit Changes:
git add . git commit -m "feat: add plane integration workflow" - Stop Development:
./scripts/stop-local.sh
Security Considerations
Credential Management
- Environment Variables:
- All sensitive data in
.env.local(not committed) - Template file
.env.local.templatefor reference - Strong passwords for local development
- All sensitive data in
- Database Security:
- Local PostgreSQL with restricted access
- No external network exposure
- Data encrypted at rest
- Network Security:
- Services only accessible via localhost
- No external port exposure
- Internal Docker network isolation
Best Practices
-
Never commit credentials:
# Add to .gitignore .env.local .env.production -
Use strong passwords:
# Generate secure passwords openssl rand -base64 32 -
Regular credential rotation:
- Change passwords monthly
- Update API tokens regularly
- Monitor access logs
Next Phase Preparation
Phase 3 Prerequisites
With local development complete, Phase 3 will focus on:
- Production Deployment - Deploy to DigitalOcean
- SSL/TLS Configuration - Traefik with Letβs Encrypt
- Monitoring Setup - Prometheus and Grafana
- Backup Systems - Automated backups
Local to Production Migration
The local development environment provides:
- Validated Workflows - All Pipeline Workflows tested locally
- Database Schema - PostgreSQL setup ready for production
- Integration Patterns - pipeline services to Plane integration proven
- Documentation - Complete setup and troubleshooting guides
Conclusion
Phase 2 establishes a robust local development environment for Zixlyβs self-hostable stack integration. This approach provides:
- Rapid Development - Local environment enables fast iteration
- Cost Efficiency - No cloud infrastructure costs during development
- Team Collaboration - Consistent local environment for all developers
- Production Readiness - Validated workflows ready for deployment
- Security - Secure credential management and network isolation
By completing Phase 2, Zixly has a fully functional local development environment with Plane integration, ready for production deployment in Phase 3.
Phase 2 Status: π― READY FOR LOCAL DEVELOPMENT
Estimated Implementation Time: 20 hours over 2 weeks
Critical Path: Docker setup β Plane configuration β Smoke test β Integration validation
Document Version: 1.0
Last Updated: 2025-01-27
Owner: Zixly Technical Architecture
Review Cycle: Daily during development