Skip to main content

Backup & Disaster Recovery — Lofty Golden Oil ERP

Version: 2.0
Last Updated: July 2026
RPO: 24 hours | RTO: 2 hours

Table of Contents​

  1. Backup Strategy Overview
  2. Database Backup
  3. Application Backup
  4. Configuration Backup
  5. Backup Schedule
  6. Backup Retention
  7. Backup Verification
  8. Recovery Procedures
  9. Partial Recovery
  10. Disaster Recovery
  11. Recovery Objectives
  12. Testing Recovery

1. Backup Strategy Overview​

3-2-1 Backup Rule​

CopiesMediaLocation
3 copies of dataOn 2 different media typesWith 1 offsite copy

Backup Components​

ComponentBackup MethodFrequencyOffsite?
MySQL databasemysqldumpDaily (2 AM)Yes (local + optional remote)
Application codeGitHub repositoryReal-time (git push)Yes (GitHub)
Uploaded filesJSON API backupDailyYes (via API)
Nginx configVersion control / manualOn changeYes (in repo)
PM2 ecosystemVersion controlOn changeYes (in repo)
SSL certificatesCertbot auto-renewalAutomaticN/A (can re-issue)
Server setupdeploy/setup.shOn changeYes (in repo)

2. Database Backup​

Manual Backup Commands​

Production Database​

# Full backup with timestamps
mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod > /tmp/lgo_prod_$(date +%Y%m%d_%H%M%S).sql

# Compressed backup (recommended for large databases)
mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod | gzip > /tmp/lgo_prod_$(date +%Y%m%d).sql.gz

# Backup with routines and events
mysqldump -u lgo -p'Lg0Erp2026!Db' --routines --events lgo_erp_prod > /tmp/lgo_prod_full_$(date +%Y%m%d).sql

# Backup specific tables only
mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod users inventory purchases > /tmp/lgo_prod_tables_$(date +%Y%m%d).sql

Staging Database​

mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_staging > /tmp/lgo_staging_$(date +%Y%m%d_%H%M%S).sql

# Compressed
mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_staging | gzip > /tmp/lgo_staging_$(date +%Y%m%d).sql.gz

All Databases​

mysqldump -u lgo -p'Lg0Erp2026!Db' --all-databases > /tmp/lgo_all_$(date +%Y%m%d).sql

Backup via ERP API​

The ERP includes a built-in JSON backup endpoint:

# Export full database as JSON
curl -s -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  https://loftygoldenoil.com/api/backup \
  -o /tmp/lgo_backup_$(date +%Y%m%d).json

# Check token validity first
curl -s https://loftygoldenoil.com/api/health

The JSON backup includes all 14 tables:

  • users, settings, inventory, purchases, production_runs
  • sales, expenses, inventory_adjustments, ar_invoices, ar_payments
  • requisitions, customers, suppliers, audit_log

Scheduled Backups via Cron​

Add to the VPS crontab:

# Edit crontab
crontab -e

# Add the following entries:
# Daily database backup at 2 AM WAT (UTC+1 = 1 AM UTC)
0 1 * * * /usr/bin/mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod | gzip > /opt/lgo-erp/backups/prod/lgo_prod_$(date +\%Y\%m\%d).sql.gz 2>/dev/null

# Daily staging backup at 2:30 AM
30 1 * * * /usr/bin/mysqldump -u lgo -p'Lg0Erp2026!Db' lgo_erp_staging | gzip > /opt/lgo-erp/backups/staging/lgo_staging_$(date +\%Y\%m\%d).sql.gz 2>/dev/null

# Cleanup backups older than 30 days
0 3 * * * find /opt/lgo-erp/backups/prod -name "*.sql.gz" -mtime +30 -delete
0 3 * * * find /opt/lgo-erp/backups/staging -name "*.sql.gz" -mtime +30 -delete

Backup Directory Setup​

# Create backup directories
sudo mkdir -p /opt/lgo-erp/backups/prod
sudo mkdir -p /opt/lgo-erp/backups/staging
sudo chown -R opc:opc /opt/lgo-erp/backups

3. Application Backup​

Source Code​

  • Stored in GitHub repository: https://github.com/abono200/lofty-golden-oil-erp.git
  • Every git push is an automatic backup
  • No additional backup needed for application code

Uploaded Files / Media​

If the system handles file uploads in the future:

# Backup any uploaded files (if applicable)
tar -czf /opt/lgo-erp/backups/uploads_$(date +%Y%m%d).tar.gz /opt/lgo-erp/uploads/

Database-Backed Files​

The ERP stores all data in MySQL. The JSON backup endpoint (/api/backup) captures all data. This is the primary backup mechanism.

4. Configuration Backup​

Nginx Configuration​

# Backup Nginx config
cp /etc/nginx/conf.d/lgo-erp.conf /opt/lgo-erp/backups/nginx_$(date +%Y%m%d).conf

# Or version control it (already in deploy/nginx.conf)
git -C /opt/lgo-erp add deploy/nginx.conf
git -C /opt/lgo-erp commit -m "Update nginx config"

PM2 Ecosystem Config​

# Already in the repo at server/ecosystem.config.cjs
# Also backed up at:
cp /opt/lgo-erp/ecosystem.config.cjs /opt/lgo-erp/backups/ecosystem_$(date +%Y%m%d).cjs

SSL Certificates​

# Certificates are managed by certbot and auto-renewed
# If lost, they can be re-issued:
sudo certbot --nginx -d loftygoldenoil.com -d www.loftygoldenoil.com

# Back up anyway (optional):
tar -czf /opt/lgo-erp/backups/ssl_$(date +%Y%m%d).tar.gz /etc/letsencrypt/live/loftygoldenoil.com/

Server Setup Script​

# The setup script at deploy/setup.sh contains the full server provisioning steps
# This is version-controlled and serves as documentation for recreation

5. Backup Schedule​

Backup TypeFrequencyTime (WAT)RetentionLocation
Production DBDaily2:00 AM30 days/opt/lgo-erp/backups/prod/
Staging DBDaily2:30 AM30 days/opt/lgo-erp/backups/staging/
Full backupWeeklySunday 3:00 AM12 months/opt/lgo-erp/backups/weekly/
Monthly archiveMonthly1st of month, 4 AMIndefinite/opt/lgo-erp/backups/monthly/
Application codeReal-timeEvery pushIndefiniteGitHub repository
Config filesOn changeManualIndefiniteGitHub + local

Weekly Full Backup​

# Add to crontab:
# Full backup every Sunday at 3 AM
0 3 * * 0 /usr/bin/mysqldump -u lgo -p'Lg0Erp2026!Db' --all-databases | gzip > /opt/lgo-erp/backups/weekly/lgo_full_$(date +\%Y\%m\%d).sql.gz 2>/dev/null

Monthly Archive​

# First of every month at 4 AM
0 4 1 * * /usr/bin/mysqldump -u lgo -p'Lg0Erp2026!Db' --all-databases | gzip > /opt/lgo-erp/backups/monthly/lgo_archive_$(date +\%Y\%m).sql.gz 2>/dev/null

6. Backup Retention​

Backup TypeRetention PeriodCleanup Method
Daily database30 daysAutomated cron (find -mtime +30 -delete)
Weekly full12 monthsManual review
Monthly archiveIndefiniteManual review
ConfigurationIndefiniteVersion control
Application codeIndefiniteGitHub

Retention Cleanup Script​

#!/bin/bash
# /opt/lgo-erp/scripts/cleanup-backups.sh

BACKUP_DIR="/opt/lgo-erp/backups"

# Remove daily backups older than 30 days
find "$BACKUP_DIR/prod" -name "*.sql.gz" -mtime +30 -delete
find "$BACKUP_DIR/staging" -name "*.sql.gz" -mtime +30 -delete

# Remove weekly backups older than 12 months
find "$BACKUP_DIR/weekly" -name "*.sql.gz" -mtime +365 -delete

# Log cleanup
echo "$(date): Backup cleanup completed" >> "$BACKUP_DIR/cleanup.log"

Make executable:

chmod +x /opt/lgo-erp/scripts/cleanup-backups.sh

7. Backup Verification​

Monthly Restore Test​

Run this procedure monthly to verify backups are valid:

Step 1: List Available Backups​

ls -la /opt/lgo-erp/backups/prod/ | tail -10

Step 2: Create Test Database​

mysql -u root -e "CREATE DATABASE IF NOT EXISTS lgo_erp_verify CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"

Step 3: Restore Backup to Test Database​

# Decompress if gzipped
gunzip -k /opt/lgo-erp/backups/prod/lgo_prod_YYYYMMDD.sql.gz

# Restore
mysql -u root lgo_erp_verify < /tmp/lgo_prod_YYYYMMDD.sql

Step 4: Verify Data Integrity​

mysql -u root lgo_erp_verify -e "
  SELECT 'users' AS tbl, COUNT(*) AS cnt FROM users
  UNION ALL SELECT 'inventory', COUNT(*) FROM inventory
  UNION ALL SELECT 'purchases', COUNT(*) FROM purchases
  UNION ALL SELECT 'production_runs', COUNT(*) FROM production_runs
  UNION ALL SELECT 'sales', COUNT(*) FROM sales
  UNION ALL SELECT 'expenses', COUNT(*) FROM expenses
  UNION ALL SELECT 'customers', COUNT(*) FROM customers
  UNION ALL SELECT 'suppliers', COUNT(*) FROM suppliers;
"

Step 5: Clean Up​

mysql -u root -e "DROP DATABASE lgo_erp_verify;"
rm /tmp/lgo_prod_YYYYMMDD.sql

Step 6: Document Results​

Record the verification result:

  • Date of backup tested
  • Tables restored successfully
  • Any data discrepancies
  • Sign-off by tester

8. Recovery Procedures​

Full Server Recovery​

Use this when the entire VPS is lost or corrupted.

Step 1: Provision New VPS​

  1. Create new Oracle Cloud instance (VM.Standard.E4.Flex, 2 OCPU, 4GB RAM)
  2. Upload SSH key
  3. Note the new public IP
  4. Update DNS A record if IP changed

Step 2: Initial Server Setup​

ssh -i /path/to/ssh-key opc@NEW_IP

# System updates
sudo dnf update -y

# Install Node.js 22
curl -fsSL https://rpm.nodesource.com/setup_22.x | sudo bash -
sudo dnf install -y nodejs

# Install MySQL
sudo dnf install -y mysql-server
sudo systemctl start mysqld
sudo systemctl enable mysqld

# Install Nginx
sudo dnf install -y nginx
sudo systemctl start nginx
sudo systemctl enable nginx

# Install PM2
sudo npm install -g pm2@7.0.3

# Install certbot
sudo dnf install -y certbot python3-certbot-nginx

Step 3: Clone Application​

cd /opt
sudo mkdir -p lgo-erp
sudo chown opc:opc lgo-erp
cd lgo-erp
git clone https://github.com/abono200/lofty-golden-oil-erp.git .
cd server
npm install --production

Step 4: Restore Database​

# Create databases and user
sudo mysql -e "
  CREATE DATABASE IF NOT EXISTS lgo_erp_prod CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
  CREATE DATABASE IF NOT EXISTS lgo_erp_staging CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
  CREATE USER IF NOT EXISTS 'lgo'@'127.0.0.1' IDENTIFIED BY 'Lg0Erp2026!Db';
  GRANT ALL PRIVILEGES ON lgo_erp_prod.* TO 'lgo'@'127.0.0.1';
  GRANT ALL PRIVILEGES ON lgo_erp_staging.* TO 'lgo'@'127.0.0.1';
  FLUSH PRIVILEGES;
"

# Restore production database from latest backup
# Download backup from your offsite location first
mysql -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod < /path/to/latest_backup.sql

# Or if compressed:
gunzip -c /path/to/latest_backup.sql.gz | mysql -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod

Step 5: Configure Nginx​

# Copy Nginx config from the repo
sudo cp /opt/lgo-erp/deploy/nginx.conf /etc/nginx/conf.d/lgo-erp.conf

# Or manually create /etc/nginx/conf.d/lgo-erp.conf
# (see Infrastructure Guide for full config)

# Comment out default server block in /etc/nginx/nginx.conf
sudo nano /etc/nginx/nginx.conf

# Test and reload
sudo nginx -t
sudo systemctl reload nginx

Step 6: Configure SELinux​

sudo setsebool -P httpd_can_network_connect on
sudo setsebool -P httpd_can_network_connect_db on

Step 7: Setup SSL​

sudo certbot --nginx -d loftygoldenoil.com -d www.loftygoldenoil.com

Step 8: Build Frontend​

cd /opt/lgo-erp
npm ci
npm run build

Step 9: Start PM2​

cd /opt/lgo-erp
pm2 start ecosystem.config.cjs
pm2 save
pm2 startup  # Follow instructions for auto-start on boot

Step 10: Create Backup Directories​

mkdir -p /opt/lgo-erp/backups/{prod,staging,weekly,monthly}

Step 11: Verify​

curl https://loftygoldenoil.com/api/health
pm2 status
pm2 logs lgo-prod --lines 10

Step 12: Update DNS​

If the IP changed:

  1. Update A record: loftygoldenoil.com → new IP
  2. Wait for propagation (5-30 minutes)
  3. Verify: dig loftygoldenoil.com +short

9. Partial Recovery​

Database Only​

# Stop the application
pm2 stop lgo-prod

# Restore database
mysql -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod < /path/to/backup.sql

# Restart application
pm2 restart lgo-prod

# Verify
curl https://loftygoldenoil.com/api/health

Application Only​

# Pull latest code
cd /opt/lgo-erp
git pull origin main

# Reinstall dependencies
cd server && npm install --production && cd ..

# Rebuild frontend
npm ci && npm run build

# Restart
pm2 restart lgo-prod

Configuration Only (Nginx/SSL)​

# Restore Nginx config
cp /opt/lgo-erp/backups/nginx_YYYYMMDD.conf /etc/nginx/conf.d/lgo-erp.conf
sudo nginx -t && sudo systemctl reload nginx

# Re-issue SSL if needed
sudo certbot --nginx -d loftygoldenoil.com -d www.loftygoldenoil.com

Restore via API (JSON Backup)​

# Get a valid JWT token first (login)
TOKEN=$(curl -s -X POST https://loftygoldenoil.com/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin123"}' | jq -r '.token')

# Restore from JSON backup
curl -X POST https://loftygoldenoil.com/api/backup/restore \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @/path/to/backup.json

Warning: This deletes all existing data and replaces it with the backup contents.

10. Disaster Recovery​

Scenario: Complete VPS Loss​

Assessment Phase (0-15 minutes)​

  1. Determine scope: Is the VPS completely lost or just down?
  2. Check Oracle Cloud: Verify instance status in the console
  3. Check backups: Verify latest backup availability
  4. Notify stakeholders: Inform the team of the outage

Recovery Phase (15-90 minutes)​

StepActionTime Estimate
1Provision new VPS5 min
2Initial server setup (Node, MySQL, Nginx, PM2)15 min
3Clone application from GitHub2 min
4Restore database from latest backup5 min
5Configure Nginx, SSL, SELinux10 min
6Build frontend and start PM25 min
7Verify functionality5 min
8Update DNS (if IP changed)5 min
Total~52 minutes

Verification Phase (90-120 minutes)​

  1. Test all user logins
  2. Verify a test transaction
  3. Check email functionality
  4. Confirm SSL certificate validity
  5. Verify DNS resolution from external network
  6. Test staging environment

Scenario: Database Corruption​

  1. Stop the application immediately:

    pm2 stop lgo-prod

  2. Assess the damage:

    mysql -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod -e "CHECK TABLE users, inventory, purchases, sales, production_runs, expenses, customers, suppliers;"

  3. Restore from latest backup:

    mysql -u lgo -p'Lg0Erp2026!Db' lgo_erp_prod < /opt/lgo-erp/backups/prod/lgo_prod_YYYYMMDD.sql

  4. Apply any transactions from audit log (manual review if needed)

  5. Restart and verify:

    pm2 restart lgo-prod
    curl https://loftygoldenoil.com/api/health

Scenario: SSL Certificate Compromise​

# Revoke old certificate
sudo certbot revoke --cert-name loftygoldenoil.com

# Issue new certificate
sudo certbot --nginx -d loftygoldenoil.com -d www.loftygoldenoil.com

# Reload Nginx
sudo systemctl reload nginx

11. Recovery Objectives​

Recovery Time Objective (RTO): 2 Hours​

The maximum acceptable downtime in a disaster scenario.

ComponentRTO TargetHow
Infrastructure30 minAutomated provisioning scripts
Database15 minmysqldump restore
Application10 mingit clone + npm ci + build
Configuration15 minVersion-controlled configs
SSL10 minCertbot re-issuance
DNS5-30 minDepends on TTL and propagation

Recovery Point Objective (RPO): 24 Hours​

The maximum acceptable data loss.

ComponentRPO TargetHow
Database24 hoursDaily backups at 2 AM
CodeReal-timeGit push (zero data loss)
ConfigurationOn changeVersion control

12. Testing Recovery​

Quarterly DR Drills​

Schedule disaster recovery drills every 3 months:

Drill Procedure​

  1. Schedule the drill (notify all stakeholders)
  2. Provision a staging VPS (separate from production)
  3. Run full recovery following the procedures above
  4. Verify all functionality:
    • Login with all 4 seed users
    • Create a test purchase, production run, and sale
    • Verify reports (Trial Balance, P&L)
    • Test email functionality
  5. Document results:
    • Total recovery time
    • Any issues encountered
    • Steps that need improvement
  6. Terminate the staging VPS

Drill Report Template​

# DR Drill Report — [DATE]

## Results
- Total Recovery Time: __ minutes
- RTO Met: Yes / No
- RPO Met: Yes / No

## Issues Encountered
- [ ] Issue 1: Description
- [ ] Issue 2: Description

## Improvements Needed
- [ ] Improvement 1
- [ ] Improvement 2

## Sign-off
- Tested by: _______________
- Reviewed by: _______________

Monthly Backup Verification​

As described in Section 7: Backup Verification, run the restore test monthly and document results.

Quick Reference​

TaskCommand
Production backupmysqldump -u lgo -p'...' lgo_erp_prod | gzip > /tmp/lgo_prod_$(date +%Y%m%d).sql.gz
Staging backupmysqldump -u lgo -p'...' lgo_erp_staging | gzip > /tmp/lgo_staging_$(date +%Y%m%d).sql.gz
Restore productiongunzip -c backup.sql.gz | mysql -u lgo -p'...' lgo_erp_prod
Restore stagingmysql -u lgo -p'...' lgo_erp_staging < backup.sql
JSON backup via APIcurl -H "Authorization: Bearer TOKEN" https://loftygoldenoil.com/api/backup > backup.json
JSON restore via APIcurl -X POST -H "Authorization: Bearer TOKEN" -d @backup.json https://loftygoldenoil.com/api/backup/restore
List backupsls -la /opt/lgo-erp/backups/prod/
Verify backupRestore to test DB → run COUNT queries