first commit
This commit is contained in:
Frank John Begornia
397
README.md
Normal file
397
README.md
Normal file
@@ -0,0 +1,397 @@
|
||||
# Shared MinIO S3 Storage
|
||||
|
||||
Centralized S3-compatible object storage service for all Crew applications.
|
||||
|
||||
## Overview
|
||||
|
||||
This MinIO instance provides S3-compatible storage shared across:
|
||||
- **crewsportswear** - Custom sportswear designs
|
||||
- **merchbay** - Merchandise platform images
|
||||
- **merchbay_admin** - Admin uploaded files
|
||||
- **crew_admin** - Admin assets
|
||||
- **email_reports** - Report attachments and images
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ crewsportswear │────▶│ │
|
||||
├─────────────────┤ │ │
|
||||
│ merchbay │────▶│ crew-minio │
|
||||
├─────────────────┤ │ │
|
||||
│ merchbay_admin │────▶│ (MinIO S3) │
|
||||
├─────────────────┤ │ │
|
||||
│ crew_admin │────▶│ │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Start MinIO
|
||||
|
||||
```bash
|
||||
cd minio-storage
|
||||
|
||||
# Local development
|
||||
docker-compose up -d
|
||||
|
||||
# Production
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
### 2. Create Buckets
|
||||
|
||||
```bash
|
||||
chmod +x setup-buckets.sh
|
||||
./setup-buckets.sh
|
||||
```
|
||||
|
||||
This creates the following buckets:
|
||||
- `crewsportswear` - Crewsportswear images
|
||||
- `merchbay` - Merchbay images
|
||||
- `merchbay-admin` - Admin uploads
|
||||
- `crew-admin` - Admin assets
|
||||
- `email-reports` - Email attachments
|
||||
|
||||
### 3. Access MinIO Console
|
||||
|
||||
**Local:** http://localhost:9001
|
||||
- Username: `minioadmin`
|
||||
- Password: `minioadmin123`
|
||||
|
||||
**Production:** https://console.crewsportswear.com
|
||||
|
||||
## Connecting Applications
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Add to each application's `.env` or `docker-compose.yml`:
|
||||
|
||||
```bash
|
||||
# Local Development (from app containers)
|
||||
MINIO_ENDPOINT=http://crew-minio:9000
|
||||
MINIO_KEY=minioadmin
|
||||
MINIO_SECRET=minioadmin123
|
||||
MINIO_BUCKET=crewsportswear # or merchbay, crew-admin, etc.
|
||||
MINIO_REGION=us-east-1
|
||||
MINIO_USE_PATH_STYLE=true
|
||||
|
||||
# Production
|
||||
MINIO_ENDPOINT=https://minio.crewsportswear.com
|
||||
MINIO_KEY=your_production_key
|
||||
MINIO_SECRET=your_production_secret
|
||||
MINIO_BUCKET=crewsportswear
|
||||
MINIO_REGION=us-east-1
|
||||
MINIO_USE_PATH_STYLE=false
|
||||
```
|
||||
|
||||
### Laravel Configuration
|
||||
|
||||
#### 1. Install AWS S3 Package
|
||||
|
||||
```bash
|
||||
composer require league/flysystem-aws-s3-v3:^1.0
|
||||
```
|
||||
|
||||
#### 2. Add to `config/filesystems.php`
|
||||
|
||||
```php
|
||||
'minio' => [
|
||||
'driver' => 's3',
|
||||
'key' => env('MINIO_KEY'),
|
||||
'secret' => env('MINIO_SECRET'),
|
||||
'region' => env('MINIO_REGION', 'us-east-1'),
|
||||
'bucket' => env('MINIO_BUCKET'),
|
||||
'endpoint' => env('MINIO_ENDPOINT'),
|
||||
'use_path_style_endpoint' => env('MINIO_USE_PATH_STYLE', true),
|
||||
],
|
||||
```
|
||||
|
||||
#### 3. Use in Code
|
||||
|
||||
```php
|
||||
use Illuminate\Support\Facades\Storage;
|
||||
|
||||
// Upload file
|
||||
Storage::disk('minio')->put('images/product.jpg', $contents);
|
||||
|
||||
// Get URL
|
||||
$url = Storage::disk('minio')->url('images/product.jpg');
|
||||
|
||||
// Check if exists
|
||||
if (Storage::disk('minio')->exists('images/product.jpg')) {
|
||||
// File exists
|
||||
}
|
||||
|
||||
// Delete file
|
||||
Storage::disk('minio')->delete('images/product.jpg');
|
||||
|
||||
// List files
|
||||
$files = Storage::disk('minio')->files('images');
|
||||
```
|
||||
|
||||
## Network Configuration
|
||||
|
||||
### Local Development
|
||||
|
||||
Apps must be on the `crew-app-net` network. Add to your app's `docker-compose.local.yml`:
|
||||
|
||||
```yaml
|
||||
networks:
|
||||
crew-app-net:
|
||||
external: true
|
||||
your-app-local:
|
||||
driver: bridge
|
||||
|
||||
services:
|
||||
app:
|
||||
networks:
|
||||
- crew-app-net # Add this
|
||||
- your-app-local
|
||||
```
|
||||
|
||||
### Create Network (First Time)
|
||||
|
||||
```bash
|
||||
docker network create crew-app-net
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
Both MinIO and your apps should be on `crew-app-net` and `traefik-public` networks (already configured).
|
||||
|
||||
## Migrating Existing Images
|
||||
|
||||
### From Old Server to MinIO
|
||||
|
||||
```bash
|
||||
# 1. Install MinIO client on your local machine
|
||||
brew install minio/stable/mc # macOS
|
||||
# OR
|
||||
wget https://dl.min.io/client/mc/release/linux-amd64/mc
|
||||
chmod +x mc
|
||||
sudo mv mc /usr/local/bin/
|
||||
|
||||
# 2. Configure MinIO alias
|
||||
mc alias set crewminio http://localhost:9000 minioadmin minioadmin123
|
||||
|
||||
# 3. Sync from old server (via rsync first)
|
||||
rsync -avz root@OLD_SERVER:/var/www/html/uploads/images/ ./temp-images/
|
||||
|
||||
# 4. Upload to MinIO
|
||||
mc cp --recursive ./temp-images/ crewminio/crewsportswear/images/
|
||||
|
||||
# 5. Verify
|
||||
mc ls crewminio/crewsportswear/images/
|
||||
|
||||
# 6. Cleanup
|
||||
rm -rf ./temp-images/
|
||||
```
|
||||
|
||||
### Direct Migration Script
|
||||
|
||||
For each app, you can use this migration pattern:
|
||||
|
||||
```bash
|
||||
# Set bucket name
|
||||
BUCKET="crewsportswear" # or merchbay, crew-admin, etc.
|
||||
|
||||
# Copy from old server directly to MinIO
|
||||
mc mirror --overwrite \
|
||||
root@OLD_SERVER:/var/www/html/uploads/images/ \
|
||||
crewminio/$BUCKET/images/
|
||||
```
|
||||
|
||||
## Production Setup
|
||||
|
||||
### 1. DNS Records
|
||||
|
||||
Create DNS records pointing to your server:
|
||||
- `minio.crewsportswear.com` → Your server IP (S3 API)
|
||||
- `console.crewsportswear.com` → Your server IP (Web Console)
|
||||
|
||||
### 2. Environment Variables
|
||||
|
||||
Create `.env` file in production:
|
||||
|
||||
```bash
|
||||
MINIO_ROOT_USER=your_secure_username
|
||||
MINIO_ROOT_PASSWORD=your_secure_password_min_8_chars
|
||||
MINIO_SERVER_URL=https://minio.crewsportswear.com
|
||||
MINIO_BROWSER_REDIRECT_URL=https://console.crewsportswear.com
|
||||
```
|
||||
|
||||
### 3. Start Service
|
||||
|
||||
```bash
|
||||
cd /var/www/minio-storage
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
### 4. Create Buckets
|
||||
|
||||
```bash
|
||||
./setup-buckets.sh
|
||||
```
|
||||
|
||||
### 5. Configure Apps
|
||||
|
||||
Update each app's production environment to point to MinIO.
|
||||
|
||||
## Backup & Restore
|
||||
|
||||
### Backup All Buckets
|
||||
|
||||
```bash
|
||||
# Backup to local directory
|
||||
mc mirror crewminio/crewsportswear ./backups/crewsportswear/
|
||||
mc mirror crewminio/merchbay ./backups/merchbay/
|
||||
mc mirror crewminio/merchbay-admin ./backups/merchbay-admin/
|
||||
mc mirror crewminio/crew-admin ./backups/crew-admin/
|
||||
|
||||
# Or backup all buckets
|
||||
for bucket in crewsportswear merchbay merchbay-admin crew-admin email-reports; do
|
||||
mc mirror crewminio/$bucket ./backups/$bucket/
|
||||
done
|
||||
```
|
||||
|
||||
### Restore Buckets
|
||||
|
||||
```bash
|
||||
mc mirror ./backups/crewsportswear/ crewminio/crewsportswear/
|
||||
```
|
||||
|
||||
### Automated Backup (Recommended)
|
||||
|
||||
Create a cron job:
|
||||
|
||||
```bash
|
||||
# /etc/cron.daily/minio-backup.sh
|
||||
#!/bin/bash
|
||||
BACKUP_DIR="/var/backups/minio/$(date +%Y%m%d)"
|
||||
mkdir -p $BACKUP_DIR
|
||||
mc mirror crewminio/ $BACKUP_DIR/
|
||||
# Keep only last 7 days
|
||||
find /var/backups/minio/ -type d -mtime +7 -exec rm -rf {} \;
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Check
|
||||
|
||||
```bash
|
||||
curl http://localhost:9000/minio/health/live
|
||||
```
|
||||
|
||||
### Storage Usage
|
||||
|
||||
```bash
|
||||
mc admin info crewminio
|
||||
```
|
||||
|
||||
### Bucket Statistics
|
||||
|
||||
```bash
|
||||
mc du crewminio/crewsportswear
|
||||
mc du crewminio/merchbay
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Refused
|
||||
|
||||
```bash
|
||||
# Check if MinIO is running
|
||||
docker ps | grep crew-minio
|
||||
|
||||
# Check logs
|
||||
docker logs crew-minio
|
||||
|
||||
# Restart
|
||||
docker restart crew-minio
|
||||
```
|
||||
|
||||
### Network Issues
|
||||
|
||||
```bash
|
||||
# Verify crew-app-net exists
|
||||
docker network ls | grep crew-app-net
|
||||
|
||||
# Create if missing
|
||||
docker network create crew-app-net
|
||||
|
||||
# Reconnect app
|
||||
docker network connect crew-app-net your-app-container
|
||||
```
|
||||
|
||||
### Permission Denied
|
||||
|
||||
```bash
|
||||
# Set public read for images
|
||||
mc anonymous set download crewminio/crewsportswear/images/
|
||||
|
||||
# Or set entire bucket public (not recommended)
|
||||
mc anonymous set public crewminio/crewsportswear/
|
||||
```
|
||||
|
||||
### Can't Access Console
|
||||
|
||||
```bash
|
||||
# Check Traefik labels (production)
|
||||
docker inspect crew-minio-prod | grep traefik
|
||||
|
||||
# Test local access
|
||||
curl http://localhost:9001
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Change default credentials** in production
|
||||
2. **Use strong passwords** (min 8 characters)
|
||||
3. **Enable HTTPS** in production (via Traefik)
|
||||
4. **Restrict bucket policies** - only make necessary paths public
|
||||
5. **Regular backups** - automate with cron
|
||||
6. **Monitor access logs** - `mc admin trace crewminio`
|
||||
7. **Use separate access keys** per application (create via console)
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### For High Traffic
|
||||
|
||||
Add to `docker-compose.prod.yml`:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
MINIO_CACHE_DRIVES: "/cache"
|
||||
MINIO_CACHE_QUOTA: 80
|
||||
```
|
||||
|
||||
### CDN Integration
|
||||
|
||||
For better performance, put CloudFlare or CloudFront in front of MinIO S3 endpoint.
|
||||
|
||||
## Cost Comparison
|
||||
|
||||
| Storage | 100GB | 500GB | 1TB |
|
||||
|---------|-------|-------|-----|
|
||||
| **MinIO (Self-hosted)** | Free* | Free* | Free* |
|
||||
| AWS S3 | $2.30/mo | $11.50/mo | $23/mo |
|
||||
| DigitalOcean Spaces | $5/mo | $15/mo | $30/mo |
|
||||
| Backblaze B2 | $0.50/mo | $2.50/mo | $5/mo |
|
||||
|
||||
*Only server costs (already running)
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ Start MinIO: `docker-compose up -d`
|
||||
2. ✅ Create buckets: `./setup-buckets.sh`
|
||||
3. ✅ Configure apps to use MinIO
|
||||
4. ✅ Migrate images from old server
|
||||
5. ✅ Test uploads/downloads
|
||||
6. ✅ Setup automated backups
|
||||
7. ✅ Delete old server
|
||||
|
||||
---
|
||||
|
||||
**MinIO Documentation:** https://min.io/docs/minio/linux/index.html
|
||||
**S3 API Reference:** https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html
|
||||
Reference in New Issue
Block a user