Skip to main content

Deployment Guide — Lofty Golden Oil ERP

Version: 2.0
Last Updated: July 2026
Environment: Production (https://loftygoldenoil.com) / Staging (http://170.9.14.60:8443)

Table of Contents​

  1. Prerequisites
  2. CI/CD Pipeline Overview
  3. How the Pipeline Works
  4. Branch Strategy
  5. Deploying to Production
  6. Deploying to Staging
  7. Manual Deployment
  8. PM2 Management
  9. Nginx Management
  10. SSL Certificate Management
  11. Environment Variables
  12. Database Migrations
  13. Rollback Procedure
  14. Monitoring
  15. Troubleshooting

1. Prerequisites​

Before deploying, ensure you have:

GitHub Repository Access​

  • Repository: https://github.com/abono200/lofty-golden-oil-erp.git
  • You need push access to main (production) and develop (staging) branches
  • Verify access: git remote -v should show the origin remote

VPS SSH Access​

  • IP: 170.9.14.60
  • User: opc
  • SSH Key: /Users/fuhsi/Documents/helpdesk/lofty-golden-oil-erp/ssh-key-2026-07-25.key
  • Test connection: 
    ssh -i /path/to/ssh-key-2026-07-25.key opc@170.9.14.60 "echo 'Connected OK'"

DNS Configured​

  • loftygoldenoil.com A record must point to 170.9.14.60
  • Verify: dig loftygoldenoil.com +short should return 170.9.14.60

GitHub Secrets Configured​

  • VPS_HOST: 170.9.14.60
  • VPS_SSH_KEY: Contents of deploy/oracle_key (private SSH key)
  • These are configured in GitHub → Settings → Secrets and variables → Actions

2. CI/CD Pipeline Overview​

The deployment pipeline is defined in .github/workflows/deploy.yml and runs automatically on every push to main.

Pipeline Architecture​

Developer pushes to main
        │
        ▼
GitHub Actions triggers
        │
        ▼
┌─────────────────┐
│  Checkout code   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Setup Node 22   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│    npm ci        │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   npm run lint   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  npm run build   │  ← Vite produces dist/
└────────┬────────┘
         │
         ▼
┌─────────────────────┐
│  SCP dist/ to VPS   │  → /opt/lgo-erp/dist/
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  SCP server files   │  → /opt/lgo-erp/server/
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│  SSH: pm2 restart   │
│  + pm2 save         │
└─────────────────────┘

3. How the Pipeline Works​

Step-by-Step Walkthrough​

Step 1 — Trigger The workflow triggers on push to main branch, or via manual workflow_dispatch.

Step 2 — Checkout

- uses: actions/checkout@v4

Clones the repository into the runner.

Step 3 — Setup Node.js 22

- uses: actions/setup-node@v4
  with:
    node-version: 22
    cache: npm

Installs Node.js 22 and caches the npm store for faster runs.

Step 4 — Install Dependencies

- run: npm ci

Clean install of all npm packages from package-lock.json.

Step 5 — Lint

- run: npm run lint

Runs ESLint to catch code quality issues before building.

Step 6 — Build

- run: npm run build

Vite compiles the React frontend into the dist/ directory (ESNext target, WASM support enabled).

Step 7 — Setup SSH

- name: Setup SSH
  run: |
    mkdir -p ~/.ssh
    echo "${{ secrets.VPS_SSH_KEY }}" > ~/.ssh/deploy_key
    chmod 600 ~/.ssh/deploy_key
    ssh-keyscan -H ${{ secrets.VPS_HOST }} >> ~/.ssh/known_hosts 2>/dev/null

Writes the private deploy key and adds the VPS host key to known_hosts.

Step 8 — Test SSH

- name: Test SSH
  run: ssh -i ~/.ssh/deploy_key -o ConnectTimeout=10 opc@${{ secrets.VPS_HOST }} "echo 'SSH OK'"

Verifies SSH connectivity before attempting deployment.

Step 9 — Deploy Frontend

- name: Deploy frontend
  run: |
    ssh -i ~/.ssh/deploy_key opc@${{ secrets.VPS_HOST }} \
      "rm -rf /opt/lgo-erp/dist/assets /opt/lgo-erp/dist/index.html /opt/lgo-erp/dist/*.svg /opt/lgo-erp/dist/*.wasm"
    scp -i ~/.ssh/deploy_key -r dist/* opc@${{ secrets.VPS_HOST }}:/opt/lgo-erp/dist/

Removes old build artifacts, then copies the new dist/ contents via SCP.

Step 10 — Deploy Server

- name: Deploy server
  run: |
    scp -i ~/.ssh/deploy_key server/server.js server/email.js opc@${{ secrets.VPS_HOST }}:/opt/lgo-erp/server/
    ssh -i ~/.ssh/deploy_key opc@${{ secrets.VPS_HOST }} \
      "pm2 restart lgo-prod && pm2 save && echo 'DEPLOYED OK'"

Copies server files, then restarts the PM2 process and saves the process list.

4. Branch Strategy​

BranchEnvironmentURLPM2 Process
mainProductionhttps://loftygoldenoil.comlgo-prod (port 4000)
developStaginghttp://170.9.14.60:8443lgo-staging (port 8443)

Rules​

  • Never push directly to main — always merge via PR from develop
  • develop is the integration branch; feature branches merge here first
  • Production deployments only happen when code is merged to main
  • Hotfixes branch from main, get merged back to both main and develop

Workflow​

feature-branch → develop → main
                           ↓
                    Production deploy

5. Deploying to Production​

# Ensure your changes are on develop
git checkout develop
git merge feature/your-feature
git push origin develop  # Deploys to staging

# After testing on staging, merge to main
git checkout main
git merge develop
git push origin main  # Triggers production deployment

Verify Deployment​

After pushing to main:

  1. Go to GitHub → Actions tab → verify the workflow completes successfully
  2. Visit https://loftygoldenoil.com and verify changes are live
  3. Check API health: curl https://loftygoldenoil.com/api/health

6. Deploying to Staging​

git checkout develop
git push origin develop

Verify Staging​

  1. Check GitHub Actions for successful workflow run
  2. Visit http://170.9.14.60:8443
  3. API health check: curl http://170.9.14.60:8443/api/health

Note: Staging runs on port 8443 directly (no Nginx SSL, no API proxy). The PM2 process serves both static files and the API.

7. Manual Deployment​

Use when GitHub Actions is unavailable or for emergency patches.

SSH into VPS​

ssh -i /path/to/ssh-key-2026-07-25.key opc@170.9.14.60
cd /opt/lgo-erp

Pull Latest Code​

git pull origin main

Install Dependencies​

cd server && npm install --production && cd ..

Build Frontend​

npm ci && npm run build

Restart PM2​

pm2 restart lgo-prod
pm2 save

Verify​

pm2 status
curl https://loftygoldenoil.com/api/health

Manual Deploy for Staging​

# Same steps but with different PM2 process
pm2 restart lgo-staging
pm2 save
curl http://170.9.14.60:8443/api/health

8. PM2 Management​

PM2 manages the Node.js processes on the VPS.

View Process Status​

pm2 status

Output shows process name, PID, CPU/memory usage, uptime, and restart count.

View Logs​

# Real-time logs
pm2 logs lgo-prod

# Last 100 lines
pm2 logs lgo-prod --lines 100

# Error logs only
pm2 logs lgo-prod --err

# Output logs only
pm2 logs lgo-prod --out

# Logs are also stored at:
# /opt/lgo-erp/server/logs/error.log
# /opt/lgo-erp/server/logs/out.log

Restart Process​

pm2 restart lgo-prod

Stop Process​

pm2 stop lgo-prod

Start Process​

pm2 start lgo-prod

Monitor Resources (Interactive)​

pm2 monit

Displays real-time CPU, memory, and network metrics for all processes.

Save Process List​

pm2 save

Saves the current process list so it can be restored after reboot.

Restore Saved Processes​

pm2 resurrect

View PM2 Configuration​

pm2 show lgo-prod

Delete Process​

pm2 delete lgo-prod

PM2 Startup (Auto-Start on Boot)​

pm2 startup
# Follow the instructions it prints
pm2 save

9. Nginx Management​

Nginx serves the React static files and proxies API requests to Node.js.

Test Configuration​

sudo nginx -t

Always run this before reloading. It checks for syntax errors.

Reload Configuration​

sudo systemctl reload nginx

Reloads config without dropping connections.

Restart Nginx​

sudo systemctl restart nginx

Check Status​

sudo systemctl status nginx

View Access Logs​

sudo tail -f /var/log/nginx/access.log

View Error Logs​

sudo tail -f /var/log/nginx/error.log

Configuration File Location​

/etc/nginx/conf.d/lgo-erp.conf

Key Nginx Configuration Details​

  • Listens on port 443 (SSL) for production
  • Serves React static files from /opt/lgo-erp/dist/
  • Proxies /api/ requests to http://127.0.0.1:4000
  • SPA fallback: try_files $uri $uri/ /index.html
  • Gotcha: The default nginx.conf server block must be commented out to avoid conflicts

10. SSL Certificate Management​

SSL is managed via Let's Encrypt with certbot.

Check Certificate Status​

sudo certbot certificates

Test Auto-Renewal​

sudo certbot renew --dry-run

Manual Renewal​

sudo certbot renew

Auto-Renewal​

Certbot installs a systemd timer that runs twice daily:

sudo systemctl status certbot-renew.timer

View Certificate Files​

/etc/letsencrypt/live/loftygoldenoil.com/
├── fullchain.pem    # Certificate chain
├── privkey.pem      # Private key
├── cert.pem         # Certificate only
└── chain.pem        # CA chain only

Revoke Certificate​

sudo certbot revoke --cert-name loftygoldenoil.com

Install New Certificate​

sudo certbot --nginx -d loftygoldenoil.com -d www.loftygoldenoil.com

11. Environment Variables​

Environment variables are set in the PM2 ecosystem config file: /opt/lgo-erp/ecosystem.config.cjs

Production Variables​

VariableValueDescription
NODE_ENVproductionRuntime environment
PORT4000Express server port
DB_HOST127.0.0.1MySQL host
DB_USERlgoMySQL username
DB_PASSLg0Erp2026!DbMySQL password
DB_NAMElgo_erp_prodMySQL database name
JWT_SECRET(see ecosystem.config.cjs)JWT signing secret
LGO_STATIC_DIR/opt/lgo-erp/distReact build output path

Staging Variables​

VariableValueDescription
NODE_ENVstagingRuntime environment
PORT8443Express server port
DB_NAMElgo_erp_stagingMySQL database name

Updating Environment Variables​

# SSH to VPS
ssh -i /path/to/ssh-key opc@170.9.14.60

# Edit the ecosystem config
nano /opt/lgo-erp/ecosystem.config.cjs

# Restart to apply
pm2 restart lgo-prod
pm2 save

Important: Never commit secrets to git. The ecosystem config on the VPS contains production secrets.

12. Database Migrations​

The system currently uses manual schema updates (no automated migration tool).

Connecting to MySQL​

# Production
mysql -u lgo -p lgo_erp_prod

# Staging
mysql -u lgo -p lgo_erp_staging

Password: Lg0Erp2026!Db

Running a Migration​

  1. Create a backup first:

    mysqldump -u lgo -p lgo_erp_prod > /tmp/lgo_prod_backup_$(date +%Y%m%d).sql

  2. Write your ALTER statements:

    -- Example: Adding a new column to customers
    ALTER TABLE customers ADD COLUMN credit_limit DOUBLE DEFAULT 0;

  3. Run against production:

    mysql -u lgo -p lgo_erp_prod < migration.sql

  4. Run against staging first:

    mysql -u lgo -p lgo_erp_staging < migration.sql

  5. Test on staging before applying to production

Schema File Location​

server/schema.sql   # Current schema definition
server/seed.sql     # Seed data

Gotcha​

The schema.sql file contains USE lgo_erp; — this must be stripped or replaced with the correct database name when applying to production or staging.

13. Rollback Procedure​

Application Rollback (Code)​

  1. Identify the last good commit:

    git log --oneline -20

  2. Revert to previous version:

    git revert HEAD
    git push origin main

    This triggers the CI/CD pipeline to deploy the previous code.

    OR reset to a specific commit:

    git reset --hard <commit-hash>
    git push --force origin main

    ⚠️ Use --force only in emergencies.

  3. Verify:

    curl https://loftygoldenoil.com/api/health

Database Rollback​

  1. Restore from backup:

    mysql -u lgo -p lgo_erp_prod < /tmp/lgo_prod_backup_YYYYMMDD.sql

  2. Or use the API backup restore:

    • Navigate to the admin panel
    • Upload a JSON backup file via the restore endpoint

Full Rollback Checklist​

  1. Revert application code
  2. Restore database from backup
  3. Restart PM2: pm2 restart lgo-prod
  4. Verify functionality
  5. Notify stakeholders

14. Monitoring​

PM2 Monitoring​

# Real-time dashboard
pm2 monit

# Quick status check
pm2 status

# Resource usage
pm2 show lgo-prod | grep -E "(memory|cpu|status)"

Log Monitoring​

# Application logs
pm2 logs lgo-prod --lines 50

# Error log file
tail -f /opt/lgo-erp/server/logs/error.log

# Nginx access log
sudo tail -f /var/log/nginx/access.log

# Nginx error log
sudo tail -f /var/log/nginx/error.log

MySQL Monitoring​

# Check MySQL status
sudo systemctl status mysqld

# Check connections
mysql -u lgo -p -e "SHOW STATUS LIKE 'Threads_connected';" lgo_erp_prod

# Check slow queries
mysql -u lgo -p -e "SHOW STATUS LIKE 'Slow_queries';" lgo_erp_prod

Health Check​

# API health
curl -s https://loftygoldenoil.com/api/health | jq .

# Expected response:
# {"status":"ok","time":"2026-07-26T...","version":2}

Uptime Monitoring​

Consider adding an external uptime monitor (e.g., UptimeRobot, Pingdom) to:

  • https://loftygoldenoil.com/api/health
  • http://170.9.14.60:8443/api/health (staging)

15. Troubleshooting​

Common Issues and Fixes​

1. Deployment Fails — SSH Connection Error​

Symptom: GitHub Actions fails at "Test SSH" step.

Fix:

  • Verify VPS_SSH_KEY secret contains the full private key (including -----BEGIN OPENSSH PRIVATE KEY-----)
  • Verify VPS_HOST is 170.9.14.60
  • Test SSH manually: ssh -i /path/to/key opc@170.9.14.60 "echo ok"

2. Site Shows 502 Bad Gateway​

Symptom: Nginx returns 502 at https://loftygoldenoil.com.

Fix:

# Check if PM2 process is running
pm2 status

# Restart if stopped
pm2 restart lgo-prod

# Check if port 4000 is listening
ss -tlnp | grep 4000

3. Static Files Not Loading (404 on JS/CSS)​

Symptom: React app loads but JS/CSS files return 404.

Fix:

  • Verify /opt/lgo-erp/dist/ contains index.html and assets/ folder
  • Check Nginx root directive: root /opt/lgo-erp/dist;
  • Rebuild and redeploy: npm run build && scp -r dist/* opc@170.9.14.60:/opt/lgo-erp/dist/

4. SSL Certificate Expired​

Symptom: Browser shows security warning.

Fix:

sudo certbot renew
sudo systemctl reload nginx

5. MySQL Connection Refused​

Symptom: API returns 500 with "Connection refused" error.

Fix:

# Check MySQL is running
sudo systemctl status mysqld

# Start if stopped
sudo systemctl start mysqld

# Check credentials
mysql -u lgo -p lgo_erp_prod

6. PM2 Process Keeps Restarting​

Symptom: pm2 status shows restarting state.

Fix:

# Check error logs
pm2 logs lgo-prod --lines 50

# Common causes:
# - Port already in use
# - Database connection failure
# - Missing environment variables

7. Port 443 Binding Error​

Symptom: Node.js crashes with "EACCES: permission denied" on port 443.

Fix: Node.js cannot bind to port 443 as non-root. Use Nginx as reverse proxy on port 443 → proxy to Node.js on port 4000. This is the current architecture.

8. SELinux Blocking Connections​

Symptom: Nginx cannot proxy to Node.js.

Fix:

sudo setsebool -P httpd_can_network_connect on
sudo setsebool -P httpd_can_network_connect_db on

9. Oracle Cloud Security List Blocks Traffic​

Symptom: Cannot access the site from external network.

Fix: Verify Oracle Cloud security list:

  • Source Port: All
  • Destination Port: 80, 443, 8443, 22
  • ⚠️ Gotcha: Do not swap source and destination ports

10. Default Nginx Server Block Conflict​

Symptom: Nginx serves default page instead of LGO ERP.

Fix:

# Comment out the default server block in /etc/nginx/nginx.conf
sudo nano /etc/nginx/nginx.conf
# Comment or remove the default `server { ... }` block
sudo nginx -t && sudo systemctl reload nginx

Quick Reference Commands​

TaskCommand
Deploy to productiongit push origin main
Deploy to staginggit push origin develop
Check PM2 statuspm2 status
Restart productionpm2 restart lgo-prod
View production logspm2 logs lgo-prod
Test Nginx configsudo nginx -t
Reload Nginxsudo systemctl reload nginx
Test SSL renewalsudo certbot renew --dry-run
Health checkcurl https://loftygoldenoil.com/api/health
SSH to VPSssh -i /path/to/key opc@170.9.14.60