webmaster/minio-storage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Frank John Begornia
2026-01-06 09:41:27 +08:00
commit c3c3e38f28
6 changed files with 855 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
.env.example Normal file
View File

@@ -0,0 +1,9 @@
# MinIO Configuration
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=minioadmin123
MINIO_SERVER_URL=http://localhost:9000
MINIO_BROWSER_REDIRECT_URL=http://localhost:9001
# Ports
MINIO_PORT=9000
MINIO_CONSOLE_PORT=9001

397
README.md Normal file
View File

@@ -0,0 +1,397 @@
# Shared MinIO S3 Storage
Centralized S3-compatible object storage service for all Crew applications.
## Overview
This MinIO instance provides S3-compatible storage shared across:
- **crewsportswear** - Custom sportswear designs
- **merchbay** - Merchandise platform images
- **merchbay_admin** - Admin uploaded files
- **crew_admin** - Admin assets
- **email_reports** - Report attachments and images
## Architecture
```
┌─────────────────┐ ┌─────────────────┐
│ crewsportswear │────▶│ │
├─────────────────┤ │ │
│ merchbay │────▶│ crew-minio │
├─────────────────┤ │ │
│ merchbay_admin │────▶│ (MinIO S3) │
├─────────────────┤ │ │
│ crew_admin │────▶│ │
└─────────────────┘ └─────────────────┘
```
## Quick Start
### 1. Start MinIO
```bash
cd minio-storage
# Local development
docker-compose up -d
# Production
docker-compose -f docker-compose.prod.yml up -d
```
### 2. Create Buckets
```bash
chmod +x setup-buckets.sh
./setup-buckets.sh
```
This creates the following buckets:
- `crewsportswear` - Crewsportswear images
- `merchbay` - Merchbay images
- `merchbay-admin` - Admin uploads
- `crew-admin` - Admin assets
- `email-reports` - Email attachments
### 3. Access MinIO Console
**Local:** http://localhost:9001
- Username: `minioadmin`
- Password: `minioadmin123`
**Production:** https://console.crewsportswear.com
## Connecting Applications
### Environment Variables
Add to each application's `.env` or `docker-compose.yml`:
```bash
# Local Development (from app containers)
MINIO_ENDPOINT=http://crew-minio:9000
MINIO_KEY=minioadmin
MINIO_SECRET=minioadmin123
MINIO_BUCKET=crewsportswear # or merchbay, crew-admin, etc.
MINIO_REGION=us-east-1
MINIO_USE_PATH_STYLE=true
# Production
MINIO_ENDPOINT=https://minio.crewsportswear.com
MINIO_KEY=your_production_key
MINIO_SECRET=your_production_secret
MINIO_BUCKET=crewsportswear
MINIO_REGION=us-east-1
MINIO_USE_PATH_STYLE=false
```
### Laravel Configuration
#### 1. Install AWS S3 Package
```bash
composer require league/flysystem-aws-s3-v3:^1.0
```
#### 2. Add to `config/filesystems.php`
```php
'minio' => [
'driver' => 's3',
'key' => env('MINIO_KEY'),
'secret' => env('MINIO_SECRET'),
'region' => env('MINIO_REGION', 'us-east-1'),
'bucket' => env('MINIO_BUCKET'),
'endpoint' => env('MINIO_ENDPOINT'),
'use_path_style_endpoint' => env('MINIO_USE_PATH_STYLE', true),
],
```
#### 3. Use in Code
```php
use Illuminate\Support\Facades\Storage;
// Upload file
Storage::disk('minio')->put('images/product.jpg', $contents);
// Get URL
$url = Storage::disk('minio')->url('images/product.jpg');
// Check if exists
if (Storage::disk('minio')->exists('images/product.jpg')) {
// File exists
}
// Delete file
Storage::disk('minio')->delete('images/product.jpg');
// List files
$files = Storage::disk('minio')->files('images');
```
## Network Configuration
### Local Development
Apps must be on the `crew-app-net` network. Add to your app's `docker-compose.local.yml`:
```yaml
networks:
crew-app-net:
external: true
your-app-local:
driver: bridge
services:
app:
networks:
- crew-app-net # Add this
- your-app-local
```
### Create Network (First Time)
```bash
docker network create crew-app-net
```
### Production
Both MinIO and your apps should be on `crew-app-net` and `traefik-public` networks (already configured).
## Migrating Existing Images
### From Old Server to MinIO
```bash
# 1. Install MinIO client on your local machine
brew install minio/stable/mc # macOS
# OR
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/
# 2. Configure MinIO alias
mc alias set crewminio http://localhost:9000 minioadmin minioadmin123
# 3. Sync from old server (via rsync first)
rsync -avz root@OLD_SERVER:/var/www/html/uploads/images/ ./temp-images/
# 4. Upload to MinIO
mc cp --recursive ./temp-images/ crewminio/crewsportswear/images/
# 5. Verify
mc ls crewminio/crewsportswear/images/
# 6. Cleanup
rm -rf ./temp-images/
```
### Direct Migration Script
For each app, you can use this migration pattern:
```bash
# Set bucket name
BUCKET="crewsportswear" # or merchbay, crew-admin, etc.
# Copy from old server directly to MinIO
mc mirror --overwrite \
root@OLD_SERVER:/var/www/html/uploads/images/ \
crewminio/$BUCKET/images/
```
## Production Setup
### 1. DNS Records
Create DNS records pointing to your server:
- `minio.crewsportswear.com` → Your server IP (S3 API)
- `console.crewsportswear.com` → Your server IP (Web Console)
### 2. Environment Variables
Create `.env` file in production:
```bash
MINIO_ROOT_USER=your_secure_username
MINIO_ROOT_PASSWORD=your_secure_password_min_8_chars
MINIO_SERVER_URL=https://minio.crewsportswear.com
MINIO_BROWSER_REDIRECT_URL=https://console.crewsportswear.com
```
### 3. Start Service
```bash
cd /var/www/minio-storage
docker-compose -f docker-compose.prod.yml up -d
```
### 4. Create Buckets
```bash
./setup-buckets.sh
```
### 5. Configure Apps
Update each app's production environment to point to MinIO.
## Backup & Restore
### Backup All Buckets
```bash
# Backup to local directory
mc mirror crewminio/crewsportswear ./backups/crewsportswear/
mc mirror crewminio/merchbay ./backups/merchbay/
mc mirror crewminio/merchbay-admin ./backups/merchbay-admin/
mc mirror crewminio/crew-admin ./backups/crew-admin/
# Or backup all buckets
for bucket in crewsportswear merchbay merchbay-admin crew-admin email-reports; do
mc mirror crewminio/$bucket ./backups/$bucket/
done
```
### Restore Buckets
```bash
mc mirror ./backups/crewsportswear/ crewminio/crewsportswear/
```
### Automated Backup (Recommended)
Create a cron job:
```bash
# /etc/cron.daily/minio-backup.sh
#!/bin/bash
BACKUP_DIR="/var/backups/minio/$(date +%Y%m%d)"
mkdir -p $BACKUP_DIR
mc mirror crewminio/ $BACKUP_DIR/
# Keep only last 7 days
find /var/backups/minio/ -type d -mtime +7 -exec rm -rf {} \;
```
## Monitoring
### Health Check
```bash
curl http://localhost:9000/minio/health/live
```
### Storage Usage
```bash
mc admin info crewminio
```
### Bucket Statistics
```bash
mc du crewminio/crewsportswear
mc du crewminio/merchbay
```
## Troubleshooting
### Connection Refused
```bash
# Check if MinIO is running
docker ps | grep crew-minio
# Check logs
docker logs crew-minio
# Restart
docker restart crew-minio
```
### Network Issues
```bash
# Verify crew-app-net exists
docker network ls | grep crew-app-net
# Create if missing
docker network create crew-app-net
# Reconnect app
docker network connect crew-app-net your-app-container
```
### Permission Denied
```bash
# Set public read for images
mc anonymous set download crewminio/crewsportswear/images/
# Or set entire bucket public (not recommended)
mc anonymous set public crewminio/crewsportswear/
```
### Can't Access Console
```bash
# Check Traefik labels (production)
docker inspect crew-minio-prod | grep traefik
# Test local access
curl http://localhost:9001
```
## Security Best Practices
1. **Change default credentials** in production
2. **Use strong passwords** (min 8 characters)
3. **Enable HTTPS** in production (via Traefik)
4. **Restrict bucket policies** - only make necessary paths public
5. **Regular backups** - automate with cron
6. **Monitor access logs** - `mc admin trace crewminio`
7. **Use separate access keys** per application (create via console)
## Performance Tuning
### For High Traffic
Add to `docker-compose.prod.yml`:
```yaml
environment:
MINIO_CACHE_DRIVES: "/cache"
MINIO_CACHE_QUOTA: 80
```
### CDN Integration
For better performance, put CloudFlare or CloudFront in front of MinIO S3 endpoint.
## Cost Comparison
| Storage | 100GB | 500GB | 1TB |
|---------|-------|-------|-----|
| **MinIO (Self-hosted)** | Free* | Free* | Free* |
| AWS S3 | $2.30/mo | $11.50/mo | $23/mo |
| DigitalOcean Spaces | $5/mo | $15/mo | $30/mo |
| Backblaze B2 | $0.50/mo | $2.50/mo | $5/mo |
*Only server costs (already running)
## Next Steps
1. ✅ Start MinIO: `docker-compose up -d`
2. ✅ Create buckets: `./setup-buckets.sh`
3. ✅ Configure apps to use MinIO
4. ✅ Migrate images from old server
5. ✅ Test uploads/downloads
6. ✅ Setup automated backups
7. ✅ Delete old server
---
**MinIO Documentation:** https://min.io/docs/minio/linux/index.html
**S3 API Reference:** https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html

277
SECURITY.md Normal file
View File

@@ -0,0 +1,277 @@
# MinIO Security Guide
## Console Access Security
### Production Console Access Options
#### ✅ Option 1: SSH Tunnel (Most Secure - RECOMMENDED)
**Don't expose console to internet at all.** Access via SSH tunnel:
```bash
# From your local machine
ssh -L 9001:localhost:9001 user@your-server
# Then access: http://localhost:9001
```
**Pros:**
- No public exposure
- No additional authentication needed
- Protected by SSH security
**Cons:**
- Requires SSH access
- Not convenient for multiple team members
---
#### ✅ Option 2: Domain with Traefik BasicAuth (Recommended for Teams)
Expose console via HTTPS with double authentication:
**Setup Steps:**
1. **Generate BasicAuth credentials:**
```bash
# Install htpasswd (if not installed)
sudo apt-get install apache2-utils # Ubuntu/Debian
# OR
brew install httpd # macOS
# Generate password hash
htpasswd -nb admin YourStrongPassword123
# Output example:
# admin:$apr1$xyz123$abc...def
```
2. **Add to production `.env`:**
```bash
# Double authentication: BasicAuth + MinIO login
TRAEFIK_CONSOLE_AUTH='admin:$$apr1$$xyz123$$abc...def' # Note: $$ for docker-compose
```
3. **Update docker-compose.prod.yml** (already done):
```yaml
- "traefik.http.routers.minio-console.middlewares=minio-auth"
- "traefik.http.middlewares.minio-auth.basicauth.users=${TRAEFIK_CONSOLE_AUTH}"
```
4. **Access:** https://console.crewsportswear.com
- First: Browser BasicAuth prompt (admin/YourStrongPassword123)
- Second: MinIO login (your MINIO_ROOT_USER/PASSWORD)
**Pros:**
- Double authentication layer
- Works from anywhere
- Can share with team
**Cons:**
- Console still publicly accessible (but protected)
- Need to manage two sets of credentials
---
#### ⚠️ Option 3: IP Whitelist (For Fixed IPs)
Add IP restriction to BasicAuth:
```yaml
# In docker-compose.prod.yml labels
- "traefik.http.middlewares.minio-ipwhitelist.ipwhitelist.sourcerange=YOUR_IP/32,OFFICE_IP/32"
- "traefik.http.routers.minio-console.middlewares=minio-auth,minio-ipwhitelist"
```
---
#### ❌ Option 4: Public Console (NOT RECOMMENDED)
**Never do this in production:**
- Exposed to brute force attacks
- Admin panel publicly accessible
- Single point of failure if credentials leaked
---
## S3 API Endpoint Security
The S3 API endpoint (`minio.crewsportswear.com`) **should remain public** for:
- Application file uploads/downloads
- Direct image access
- S3 API operations
**Why it's safe:**
- Requires valid access keys
- Bucket policies control access
- Can't browse/delete without credentials
**Example public URL:**
```
https://minio.crewsportswear.com/crewsportswear/images/product.jpg
```
Only publicly readable paths (like `/images/`) are accessible. Private uploads require auth.
---
## Production Security Checklist
### 1. Strong Credentials
```bash
# Generate strong password
openssl rand -base64 32
# Update .env
MINIO_ROOT_USER=admin_$(date +%s) # Unique username
MINIO_ROOT_PASSWORD=<generated_password>
```
### 2. Console Access
- [ ] Use SSH tunnel OR
- [ ] Enable Traefik BasicAuth
- [ ] Consider IP whitelist
- [ ] Never expose without protection
### 3. Bucket Policies
```bash
# Only make necessary paths public
mc anonymous set download crewminio/crewsportswear/images/
# Keep uploads private
mc anonymous set none crewminio/crewsportswear/uploads/
```
### 4. Application Access Keys
Create separate access keys per app (don't use root credentials):
```bash
# Access MinIO console → Administrator → Users → Create User
# Or via mc:
mc admin user add crewminio crewsportswear-app <password>
mc admin policy attach crewminio readwrite --user crewsportswear-app
```
Then use in app:
```bash
MINIO_KEY=crewsportswear-app
MINIO_SECRET=<password>
```
### 5. Network Segmentation
```yaml
# Apps should access MinIO via internal network
MINIO_ENDPOINT=http://crew-minio:9000 # Internal
# Not: https://minio.crewsportswear.com (external)
```
### 6. HTTPS Only
```yaml
environment:
- MINIO_SERVER_URL=https://minio.crewsportswear.com # Force HTTPS
```
### 7. Regular Backups
```bash
# Automated daily backup
mc mirror crewminio/ /var/backups/minio/$(date +%Y%m%d)/
```
### 8. Monitor Access Logs
```bash
# Enable audit logging
docker exec crew-minio-prod mc admin trace crewminio
```
---
## Recommended Production Setup
### For Solo Developer:
```yaml
# Don't expose console publicly
# Remove console Traefik labels
# Access via SSH tunnel only
```
### For Small Team (2-5 people):
```yaml
# Use BasicAuth + MinIO login (double auth)
# IP whitelist to office/VPN
# Strong passwords (32+ chars)
```
### For Larger Team:
```yaml
# Use VPN (Tailscale/WireGuard) + internal console
# Separate access keys per app/user
# Audit logging enabled
# Regular security reviews
```
---
## Quick Setup Commands
### Secure Console (BasicAuth)
```bash
# 1. Generate password
htpasswd -nb admin $(openssl rand -base64 16)
# Output: admin:$apr1$xyz...
# Copy this to .env
# 2. Update .env
cat >> .env << EOF
TRAEFIK_CONSOLE_AUTH='admin:\$\$apr1\$\$xyz...' # Escape $ with $$
EOF
# 3. Restart
docker-compose -f docker-compose.prod.yml up -d
```
### SSH Tunnel (No Public Console)
```bash
# 1. Remove console Traefik labels from docker-compose.prod.yml
# 2. Don't expose port 9001
# 3. Access via SSH:
ssh -L 9001:localhost:9001 user@server
# Then: http://localhost:9001
```
---
## Emergency: Credentials Compromised
```bash
# 1. Immediately change root password
docker exec crew-minio-prod mc admin user update crewminio minioadmin --password NewPassword123
# 2. Revoke leaked access keys
docker exec crew-minio-prod mc admin user disable crewminio leaked-user
# 3. Review access logs
docker exec crew-minio-prod mc admin trace crewminio
# 4. Rotate application keys
# Update all apps with new credentials
# 5. Check for unauthorized files
docker exec crew-minio-prod mc ls --recursive crewminio/
```
---
## Summary
| Access Method | Security Level | Use Case |
|--------------|----------------|----------|
| SSH Tunnel | ⭐⭐⭐⭐⭐ | Solo dev, maximum security |
| BasicAuth + IP | ⭐⭐⭐⭐ | Small team, fixed IPs |
| BasicAuth Only | ⭐⭐⭐ | Remote team, can't use VPN |
| Public Console | ⭐ | **NEVER USE** |
**Recommendation:** Start with SSH tunnel. Add BasicAuth only if multiple people need access.

62
docker-compose.prod.yml Normal file
View File

@@ -0,0 +1,62 @@
version: '3.8'
services:
minio:
image: minio/minio:RELEASE.2024-10-02T17-50-41Z
container_name: crew-minio-prod
restart: unless-stopped
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
MINIO_SERVER_URL: ${MINIO_SERVER_URL:-https://minio.crewsportswear.com}
MINIO_BROWSER_REDIRECT_URL: ${MINIO_BROWSER_REDIRECT_URL:-https://console.crewsportswear.com}
command: server /data --console-address ":9001"
volumes:
- minio-data:/data
networks:
- traefik-public
- crew-app-net
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
labels:
- "traefik.enable=true"
# MinIO API (S3 endpoint)
- "traefik.http.routers.minio-api.rule=Host(`minio.crewsportswear.com`)"
- "traefik.http.routers.minio-api.entrypoints=websecure"
- "traefik.http.routers.minio-api.tls=true"
- "traefik.http.routers.minio-api.service=minio-api"
- "traefik.http.services.minio-api.loadbalancer.server.port=9000"
# MinIO Console (Web UI) - SECURED WITH BASIC AUTH
- "traefik.http.routers.minio-console.rule=Host(`console.crewsportswear.com`)"
- "traefik.http.routers.minio-console.entrypoints=websecure"
- "traefik.http.routers.minio-console.tls=true"
- "traefik.http.routers.minio-console.service=minio-console"
- "traefik.http.routers.minio-console.middlewares=minio-auth"
- "traefik.http.services.minio-console.loadbalancer.server.port=9001"
# Basic Auth Middleware (Generate with: htpasswd -nb admin yourpassword)
# Example: admin:$apr1$xyz... (replace with your own)
- "traefik.http.middlewares.minio-auth.basicauth.users=${TRAEFIK_CONSOLE_AUTH}"
# HTTP to HTTPS redirect
- "traefik.http.routers.minio-api-http.rule=Host(`minio.crewsportswear.com`)"
- "traefik.http.routers.minio-api-http.entrypoints=web"
- "traefik.http.routers.minio-api-http.middlewares=https-redirect"
- "traefik.http.routers.minio-console-http.rule=Host(`console.crewsportswear.com`)"
- "traefik.http.routers.minio-console-http.entrypoints=web"
- "traefik.http.routers.minio-console-http.middlewares=https-redirect"
networks:
traefik-public:
external: true
crew-app-net:
external: true
volumes:
minio-data:
driver: local

35
docker-compose.yml Normal file
View File

@@ -0,0 +1,35 @@
version: '3.8'
services:
minio:
image: minio/minio:RELEASE.2024-10-02T17-50-41Z
container_name: crew-minio
restart: unless-stopped
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER:-minioadmin}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD:-minioadmin123}
MINIO_SERVER_URL: ${MINIO_SERVER_URL:-http://localhost:9000}
MINIO_BROWSER_REDIRECT_URL: ${MINIO_BROWSER_REDIRECT_URL:-http://localhost:9001}
command: server /data --console-address ":9001"
ports:
- "${MINIO_PORT:-9000}:9000"
- "${MINIO_CONSOLE_PORT:-9001}:9001"
volumes:
- minio-data:/data
networks:
- crew-app-net
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
labels:
- "description=Shared MinIO S3-compatible storage for all Crew apps"
networks:
crew-app-net:
external: true
volumes:
minio-data:
driver: local

75
setup-buckets.sh Normal file
View File

@@ -0,0 +1,75 @@
#!/bin/bash
# Setup MinIO buckets for all Crew applications
set -e
MINIO_ALIAS="crewminio"
MINIO_ENDPOINT="http://crew-minio:9000"
MINIO_USER="${MINIO_ROOT_USER:-minioadmin}"
MINIO_PASSWORD="${MINIO_ROOT_PASSWORD:-minioadmin123}"
echo "=========================================="
echo "Setting up MinIO buckets for Crew apps"
echo "=========================================="
echo ""
# Check if MinIO is running
if ! docker ps | grep -q crew-minio; then
echo "❌ Error: crew-minio container is not running"
echo " Start it first: docker-compose up -d"
exit 1
fi
echo "Installing MinIO Client (mc)..."
docker exec crew-minio sh -c "
if ! command -v mc &> /dev/null; then
curl -O https://dl.min.io/client/mc/release/linux-amd64/mc && \
chmod +x mc && \
mv mc /usr/local/bin/
fi
"
echo "✓ MinIO Client installed"
echo ""
# Configure MinIO client
echo "Configuring MinIO client..."
docker exec crew-minio mc alias set $MINIO_ALIAS $MINIO_ENDPOINT $MINIO_USER $MINIO_PASSWORD
echo ""
# Create buckets for each application
BUCKETS=("crewsportswear" "merchbay" "merchbay-admin" "crew-admin" "email-reports")
for BUCKET in "${BUCKETS[@]}"; do
echo "Creating bucket: $BUCKET"
docker exec crew-minio mc mb $MINIO_ALIAS/$BUCKET --ignore-existing
# Set public read policy for images folder
echo " ↳ Setting public read access for $BUCKET/images/"
docker exec crew-minio mc anonymous set download $MINIO_ALIAS/$BUCKET/images/
echo " ✓ Bucket $BUCKET ready"
echo ""
done
echo "=========================================="
echo "✓ All buckets created successfully!"
echo "=========================================="
echo ""
echo "Buckets created:"
for BUCKET in "${BUCKETS[@]}"; do
echo " - $BUCKET"
done
echo ""
echo "Access MinIO Console:"
echo " Local: http://localhost:9001"
echo " Production: https://console.crewsportswear.com"
echo ""
echo "Credentials:"
echo " Username: $MINIO_USER"
echo " Password: $MINIO_PASSWORD"
echo ""
echo "S3 Endpoint:"
echo " Local: http://crew-minio:9000 (from app containers)"
echo " http://localhost:9000 (from host)"
echo " Production: https://minio.crewsportswear.com"