Initial commit of Forgejo configuration
This commit is contained in:
Dada
commit
ce9b28917b
16 changed files with 860 additions and 0 deletions
27
.gitignore
vendored
Normal file
27
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
# Terraform files
|
||||||
|
terraform/.terraform/
|
||||||
|
terraform/terraform.tfstate
|
||||||
|
terraform/terraform.tfstate.backup
|
||||||
|
terraform/.terraform.lock.hcl
|
||||||
|
terraform/crash.log
|
||||||
|
|
||||||
|
# Ansible files
|
||||||
|
ansible/*.retry
|
||||||
|
|
||||||
|
# SSH keys
|
||||||
|
*.pem
|
||||||
|
*.key
|
||||||
|
id_rsa*
|
||||||
|
|
||||||
|
# OS files
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Editor files
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*~
|
||||||
|
|
||||||
|
# Logs
|
||||||
|
*.log
|
||||||
147
DEPLOYMENT_GUIDE.md
Normal file
147
DEPLOYMENT_GUIDE.md
Normal file
|
|
@ -0,0 +1,147 @@
|
||||||
|
# Forgejo Deployment Guide
|
||||||
|
|
||||||
|
This guide provides detailed step-by-step instructions for deploying the Forgejo git repository on a GCP VM with SSL.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
1. Google Cloud Platform account with project ID: `homelab-454516`
|
||||||
|
2. Terraform installed locally
|
||||||
|
3. Ansible installed locally
|
||||||
|
4. SSH key pair at `~/.ssh/id_rsa` and `~/.ssh/id_rsa.pub`
|
||||||
|
5. DuckDNS account with subdomain: `bootknockery.duckdns.org`
|
||||||
|
|
||||||
|
## Deployment Steps
|
||||||
|
|
||||||
|
### 1. Infrastructure Provisioning with Terraform
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Navigate to the terraform directory
|
||||||
|
cd terraform
|
||||||
|
|
||||||
|
# Initialize Terraform
|
||||||
|
terraform init
|
||||||
|
|
||||||
|
# Apply the Terraform configuration
|
||||||
|
terraform apply
|
||||||
|
```
|
||||||
|
|
||||||
|
This will create a GCP VM in the `us-central1-a` zone with the following specifications:
|
||||||
|
- Machine type: e2-medium
|
||||||
|
- Disk size: 50GB
|
||||||
|
- Operating system: Debian
|
||||||
|
|
||||||
|
### 2. Update DuckDNS
|
||||||
|
|
||||||
|
Ensure your DuckDNS subdomain (`bootknockery.duckdns.org`) points to the VM's external IP address.
|
||||||
|
|
||||||
|
### 3. Deploy Forgejo with Ansible
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Navigate to the ansible directory
|
||||||
|
cd ../ansible
|
||||||
|
|
||||||
|
# Run the Ansible playbook
|
||||||
|
ansible-playbook -i inventory.yml forgejo.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
- Install Docker and Docker Compose
|
||||||
|
- Deploy Forgejo in a Docker container
|
||||||
|
- Configure Nginx as a reverse proxy
|
||||||
|
- Set up SSL with Let's Encrypt
|
||||||
|
- Configure SQLite with WAL mode for better performance
|
||||||
|
|
||||||
|
### 4. Verify the Deployment
|
||||||
|
|
||||||
|
1. Access Forgejo via HTTPS:
|
||||||
|
```
|
||||||
|
https://bootknockery.duckdns.org
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Check that SSL is properly configured:
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo certbot certificates"
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Verify that SQLite WAL mode is active:
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo ls -la /opt/forgejo/data/gitea/gitea.db*"
|
||||||
|
```
|
||||||
|
You should see additional files like `gitea.db-wal` and `gitea.db-shm`.
|
||||||
|
|
||||||
|
## Post-Deployment Configuration
|
||||||
|
|
||||||
|
### Initial Forgejo Setup
|
||||||
|
|
||||||
|
When you first access Forgejo, you'll need to:
|
||||||
|
|
||||||
|
1. Set up the admin account
|
||||||
|
2. Configure the database settings (SQLite is already configured)
|
||||||
|
3. Set up repository settings
|
||||||
|
|
||||||
|
### SSH Access for Git Operations
|
||||||
|
|
||||||
|
To use Git over SSH:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Add to ~/.ssh/config
|
||||||
|
Host bootknockery.duckdns.org
|
||||||
|
Port 222
|
||||||
|
User git
|
||||||
|
IdentityFile ~/.ssh/id_rsa
|
||||||
|
```
|
||||||
|
|
||||||
|
Then clone repositories using:
|
||||||
|
```bash
|
||||||
|
git clone git@bootknockery.duckdns.org:username/repository.git
|
||||||
|
```
|
||||||
|
|
||||||
|
## Maintenance Tasks
|
||||||
|
|
||||||
|
### SSL Certificate Renewal
|
||||||
|
|
||||||
|
SSL certificates are automatically renewed weekly. To manually renew:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo /etc/cron.weekly/certbot-renew"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Backup
|
||||||
|
|
||||||
|
To back up the Forgejo database:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo cp /opt/forgejo/data/gitea/gitea.db ~/backup_gitea.db"
|
||||||
|
scp -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS:~/backup_gitea.db ./
|
||||||
|
```
|
||||||
|
|
||||||
|
### Updating Forgejo
|
||||||
|
|
||||||
|
To update Forgejo to a newer version:
|
||||||
|
|
||||||
|
1. Edit `ansible/templates/docker-compose.yml.j2` to update the image version
|
||||||
|
2. Run the Ansible playbook again:
|
||||||
|
```bash
|
||||||
|
cd ansible
|
||||||
|
ansible-playbook -i inventory.yml forgejo.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
1. **Cannot access Forgejo via HTTPS**
|
||||||
|
- Check Nginx status: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo systemctl status nginx"`
|
||||||
|
- Verify SSL certificates: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo ls -la /etc/letsencrypt/live/bootknockery.duckdns.org/"`
|
||||||
|
|
||||||
|
2. **Forgejo container not running**
|
||||||
|
- Check Docker status: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo docker ps"`
|
||||||
|
- View Forgejo logs: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo docker logs forgejo"`
|
||||||
|
|
||||||
|
3. **SSL certificate issues**
|
||||||
|
- Check certificate logs: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo tail -f /var/log/letsencrypt/letsencrypt.log"`
|
||||||
|
- Force renewal: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo /etc/cron.weekly/certbot-renew"`
|
||||||
|
|
||||||
|
4. **Performance issues**
|
||||||
|
- Verify WAL mode: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo ls -la /opt/forgejo/data/gitea/gitea.db*"`
|
||||||
|
- Check system resources: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "top"`
|
||||||
142
README.md
Normal file
142
README.md
Normal file
|
|
@ -0,0 +1,142 @@
|
||||||
|
# GCP Data Engineering VM with Forgejo
|
||||||
|
|
||||||
|
This project sets up a reusable Google Cloud Platform (GCP) virtual machine for data engineering and machine learning projects, with Forgejo as the first installed module.
|
||||||
|
|
||||||
|
## Technologies Used
|
||||||
|
|
||||||
|
- **Terraform**: For infrastructure provisioning and state management
|
||||||
|
- **Cloud-init**: For initial VM setup and configuration with Debian
|
||||||
|
- **Ansible**: For configuration management and application deployment
|
||||||
|
- **Docker & Docker Compose**: For containerized applications
|
||||||
|
- **Forgejo**: Open source git repository that can be accessed through a DuckDNS subdomain
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
.
|
||||||
|
├── ansible/ # Ansible playbooks and configurations
|
||||||
|
│ ├── forgejo.yml # Forgejo installation playbook
|
||||||
|
│ ├── inventory.yml # Ansible inventory file
|
||||||
|
│ └── templates/ # Jinja2 templates for configurations
|
||||||
|
│ ├── app.ini.j2 # Forgejo app configuration
|
||||||
|
│ ├── docker-compose.yml.j2 # Docker Compose template
|
||||||
|
│ ├── nginx.conf.j2 # Nginx configuration with SSL support
|
||||||
|
│ └── certbot-renew.j2 # SSL certificate renewal script
|
||||||
|
├── cloud-init/ # Cloud-init scripts
|
||||||
|
│ └── cloud-init.sh # Initial VM setup script
|
||||||
|
├── terraform/ # Terraform configurations
|
||||||
|
│ ├── main.tf # Main Terraform configuration
|
||||||
|
│ ├── variables.tf # Variable definitions
|
||||||
|
│ └── terraform.tfvars.example # Example variable values
|
||||||
|
├── deploy.sh # Deployment automation script
|
||||||
|
└── README.md # This file
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
1. Google Cloud Platform account with a project
|
||||||
|
2. Terraform installed locally
|
||||||
|
3. Ansible installed locally
|
||||||
|
4. SSH key pair for VM access
|
||||||
|
5. DuckDNS subdomain for Forgejo access
|
||||||
|
|
||||||
|
## Setup Instructions
|
||||||
|
|
||||||
|
### 1. Configure Terraform Variables
|
||||||
|
|
||||||
|
Copy the example variables file and edit it with your GCP project details:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp terraform/terraform.tfvars.example terraform/terraform.tfvars
|
||||||
|
```
|
||||||
|
|
||||||
|
Edit `terraform/terraform.tfvars` with your GCP project ID, preferred region/zone, and SSH key path.
|
||||||
|
|
||||||
|
### 2. Configure Ansible Inventory
|
||||||
|
|
||||||
|
Edit `ansible/inventory.yml` to set your DuckDNS subdomain and admin email for Let's Encrypt SSL certificates.
|
||||||
|
|
||||||
|
### 3. Run the Deployment Script
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This script will:
|
||||||
|
- Initialize and apply Terraform configuration to create the VM
|
||||||
|
- Update the Ansible inventory with the VM's IP address
|
||||||
|
- Wait for the VM to be ready
|
||||||
|
- Run the Ansible playbook to install and configure Forgejo
|
||||||
|
|
||||||
|
## Accessing Forgejo
|
||||||
|
|
||||||
|
Once deployment is complete, Forgejo will be accessible at:
|
||||||
|
|
||||||
|
```
|
||||||
|
https://your-duckdns-subdomain.duckdns.org
|
||||||
|
```
|
||||||
|
|
||||||
|
The deployment automatically configures:
|
||||||
|
- HTTPS with Let's Encrypt SSL certificates
|
||||||
|
- Automatic HTTP to HTTPS redirection
|
||||||
|
- Weekly SSL certificate renewal
|
||||||
|
- SQLite with WAL mode for improved performance
|
||||||
|
|
||||||
|
SSH access to the Forgejo Git server will be available on port 222:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh -p 222 git@your-duckdns-subdomain.duckdns.org
|
||||||
|
```
|
||||||
|
|
||||||
|
## Performance Optimizations
|
||||||
|
|
||||||
|
### SQLite WAL Mode
|
||||||
|
|
||||||
|
The deployment configures SQLite to use Write-Ahead Logging (WAL) mode for better performance. This provides:
|
||||||
|
- Improved write performance with concurrent operations
|
||||||
|
- Better read concurrency (readers don't block writers)
|
||||||
|
- Reduced disk I/O
|
||||||
|
- Improved durability and crash recovery
|
||||||
|
|
||||||
|
To modify this setting, edit the `ansible/templates/app.ini.j2` file:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[database]
|
||||||
|
SQLITE_JOURNAL_MODE = WAL
|
||||||
|
```
|
||||||
|
|
||||||
|
## Customization
|
||||||
|
|
||||||
|
- **Machine Type**: Edit `terraform/terraform.tfvars` to change the VM size
|
||||||
|
- **Forgejo Configuration**: Modify `ansible/templates/app.ini.j2` to customize Forgejo settings
|
||||||
|
- **Additional Applications**: Add new Ansible playbooks in the `ansible` directory
|
||||||
|
|
||||||
|
## Maintenance
|
||||||
|
|
||||||
|
- **Terraform State**: The Terraform state is stored locally in `terraform/terraform.tfstate`
|
||||||
|
- **VM Updates**: SSH into the VM and run standard Debian update commands
|
||||||
|
- **Forgejo Updates**: Update the Docker image version in `ansible/templates/docker-compose.yml.j2`
|
||||||
|
- **SSL Certificate Renewal**: Certificates are automatically renewed weekly via a cron job
|
||||||
|
- **Database Backups**: To back up the Forgejo database, copy `/opt/forgejo/data/gitea/gitea.db*` from the VM
|
||||||
|
- **Monitoring**: Check the status of services with:
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo systemctl status nginx"
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo docker ps"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
- **SSH Access**: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS`
|
||||||
|
- **Logs**:
|
||||||
|
- Forgejo logs: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo docker logs forgejo"`
|
||||||
|
- Nginx logs: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo tail -f /var/log/nginx/error.log"`
|
||||||
|
- SSL certificate logs: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo tail -f /var/log/letsencrypt/letsencrypt.log"`
|
||||||
|
- **Connectivity**: Ensure GCP firewall rules allow traffic on ports 80, 443, 22, 3000, and 222
|
||||||
|
- **SSL Issues**:
|
||||||
|
- Check certificate status: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo certbot certificates"`
|
||||||
|
- Force certificate renewal: `ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo /etc/cron.weekly/certbot-renew"`
|
||||||
|
- **Database Performance**: If experiencing slowdowns, check SQLite WAL mode is working:
|
||||||
|
```bash
|
||||||
|
ssh -i ~/.ssh/id_rsa debian@VM_IP_ADDRESS "sudo ls -la /opt/forgejo/data/gitea/gitea.db*"
|
||||||
|
```
|
||||||
|
You should see additional files like `gitea.db-wal` and `gitea.db-shm` if WAL mode is active.
|
||||||
124
ansible/forgejo.yml
Normal file
124
ansible/forgejo.yml
Normal file
|
|
@ -0,0 +1,124 @@
|
||||||
|
---
|
||||||
|
- name: Install and configure Forgejo
|
||||||
|
hosts: all
|
||||||
|
become: true
|
||||||
|
vars:
|
||||||
|
forgejo_domain: "{{ duckdns_subdomain }}.duckdns.org"
|
||||||
|
forgejo_data_dir: /opt/forgejo
|
||||||
|
forgejo_user_uid: 1000
|
||||||
|
forgejo_user_gid: 1000
|
||||||
|
forgejo_http_port: 3000
|
||||||
|
forgejo_ssh_port: 222
|
||||||
|
|
||||||
|
tasks:
|
||||||
|
- name: Ensure Forgejo directory exists
|
||||||
|
file:
|
||||||
|
path: "{{ forgejo_data_dir }}"
|
||||||
|
state: directory
|
||||||
|
owner: "{{ forgejo_user_uid }}"
|
||||||
|
group: "{{ forgejo_user_gid }}"
|
||||||
|
mode: '0755'
|
||||||
|
|
||||||
|
- name: Create Forgejo docker-compose.yml
|
||||||
|
template:
|
||||||
|
src: templates/docker-compose.yml.j2
|
||||||
|
dest: "{{ forgejo_data_dir }}/docker-compose.yml"
|
||||||
|
owner: "{{ forgejo_user_uid }}"
|
||||||
|
group: "{{ forgejo_user_gid }}"
|
||||||
|
mode: '0644'
|
||||||
|
|
||||||
|
- name: Create app.ini configuration file
|
||||||
|
template:
|
||||||
|
src: templates/app.ini.j2
|
||||||
|
dest: "{{ forgejo_data_dir }}/app.ini"
|
||||||
|
owner: "{{ forgejo_user_uid }}"
|
||||||
|
group: "{{ forgejo_user_gid }}"
|
||||||
|
mode: '0644'
|
||||||
|
|
||||||
|
- name: Start Forgejo with Docker Compose
|
||||||
|
community.docker.docker_compose_v2:
|
||||||
|
project_src: "{{ forgejo_data_dir }}"
|
||||||
|
state: present
|
||||||
|
become: true
|
||||||
|
become_user: debian
|
||||||
|
|
||||||
|
- name: Install Nginx
|
||||||
|
apt:
|
||||||
|
name: nginx
|
||||||
|
state: present
|
||||||
|
update_cache: yes
|
||||||
|
|
||||||
|
- name: Configure Nginx for Forgejo
|
||||||
|
template:
|
||||||
|
src: templates/nginx.conf.j2
|
||||||
|
dest: /etc/nginx/sites-available/forgejo
|
||||||
|
owner: root
|
||||||
|
group: root
|
||||||
|
mode: '0644'
|
||||||
|
|
||||||
|
- name: Enable Nginx site
|
||||||
|
file:
|
||||||
|
src: /etc/nginx/sites-available/forgejo
|
||||||
|
dest: /etc/nginx/sites-enabled/forgejo
|
||||||
|
state: link
|
||||||
|
|
||||||
|
- name: Remove default Nginx site
|
||||||
|
file:
|
||||||
|
path: /etc/nginx/sites-enabled/default
|
||||||
|
state: absent
|
||||||
|
|
||||||
|
- name: Restart Nginx
|
||||||
|
service:
|
||||||
|
name: nginx
|
||||||
|
state: restarted
|
||||||
|
enabled: yes
|
||||||
|
|
||||||
|
- name: Pull Certbot Docker image
|
||||||
|
community.docker.docker_image:
|
||||||
|
name: certbot/certbot
|
||||||
|
source: pull
|
||||||
|
when: admin_email is defined
|
||||||
|
|
||||||
|
- name: Stop Nginx before obtaining SSL certificate
|
||||||
|
service:
|
||||||
|
name: nginx
|
||||||
|
state: stopped
|
||||||
|
when: admin_email is defined
|
||||||
|
|
||||||
|
- name: Obtain SSL certificate with Certbot Docker
|
||||||
|
block:
|
||||||
|
- name: Run Certbot Docker to obtain SSL certificate
|
||||||
|
command: >
|
||||||
|
docker run --rm -p 80:80 -p 443:443
|
||||||
|
-v /etc/letsencrypt:/etc/letsencrypt
|
||||||
|
-v /var/lib/letsencrypt:/var/lib/letsencrypt
|
||||||
|
certbot/certbot certonly --standalone
|
||||||
|
-d {{ forgejo_domain }} --non-interactive --agree-tos
|
||||||
|
-m {{ admin_email }}
|
||||||
|
args:
|
||||||
|
creates: /etc/letsencrypt/live/{{ forgejo_domain }}/fullchain.pem
|
||||||
|
register: certbot_output
|
||||||
|
failed_when: certbot_output.rc != 0 and certbot_output.stderr is not search("already exists")
|
||||||
|
- name: Handle certbot errors
|
||||||
|
debug:
|
||||||
|
msg: "Certbot error: {{ certbot_output.stderr }}"
|
||||||
|
when: certbot_output is failed
|
||||||
|
when: admin_email is defined
|
||||||
|
ignore_errors: yes
|
||||||
|
|
||||||
|
- name: Start Nginx after obtaining SSL certificate
|
||||||
|
service:
|
||||||
|
name: nginx
|
||||||
|
state: started
|
||||||
|
when: admin_email is defined
|
||||||
|
|
||||||
|
- name: Set up SSL certificate renewal
|
||||||
|
block:
|
||||||
|
- name: Create SSL certificate renewal script
|
||||||
|
template:
|
||||||
|
src: templates/certbot-renew.j2
|
||||||
|
dest: /etc/cron.weekly/certbot-renew
|
||||||
|
owner: root
|
||||||
|
group: root
|
||||||
|
mode: '0755'
|
||||||
|
when: admin_email is defined
|
||||||
8
ansible/inventory.yml
Normal file
8
ansible/inventory.yml
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
all:
|
||||||
|
hosts:
|
||||||
|
data_engineering_vm:
|
||||||
|
ansible_host: "34.67.170.119"
|
||||||
|
ansible_user: debian
|
||||||
|
ansible_ssh_private_key_file: "~/.ssh/id_rsa"
|
||||||
|
duckdns_subdomain: "bootknockery" # Replace with your actual DuckDNS subdomain
|
||||||
|
admin_email: "jaypeedaylee@gmail.com" # Replace with your email for Let's Encrypt
|
||||||
77
ansible/templates/app.ini.j2
Normal file
77
ansible/templates/app.ini.j2
Normal file
|
|
@ -0,0 +1,77 @@
|
||||||
|
APP_NAME = Forgejo
|
||||||
|
RUN_USER = git
|
||||||
|
RUN_MODE = prod
|
||||||
|
|
||||||
|
[repository]
|
||||||
|
ROOT = /data/git/repositories
|
||||||
|
|
||||||
|
[repository.local]
|
||||||
|
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo
|
||||||
|
|
||||||
|
[repository.upload]
|
||||||
|
TEMP_PATH = /data/gitea/uploads
|
||||||
|
|
||||||
|
[server]
|
||||||
|
APP_DATA_PATH = /data/gitea
|
||||||
|
DOMAIN = {{ forgejo_domain }}
|
||||||
|
SSH_DOMAIN = {{ forgejo_domain }}
|
||||||
|
HTTP_PORT = 3000
|
||||||
|
ROOT_URL = https://{{ forgejo_domain }}/
|
||||||
|
DISABLE_SSH = false
|
||||||
|
SSH_PORT = {{ forgejo_ssh_port }}
|
||||||
|
SSH_LISTEN_PORT = 22
|
||||||
|
LFS_START_SERVER = true
|
||||||
|
LFS_CONTENT_PATH = /data/git/lfs
|
||||||
|
LFS_JWT_SECRET =
|
||||||
|
OFFLINE_MODE = false
|
||||||
|
|
||||||
|
[database]
|
||||||
|
PATH = /data/gitea/gitea.db
|
||||||
|
DB_TYPE = sqlite3
|
||||||
|
SQLITE_JOURNAL_MODE = WAL
|
||||||
|
|
||||||
|
[indexer]
|
||||||
|
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve
|
||||||
|
|
||||||
|
[session]
|
||||||
|
PROVIDER_CONFIG = /data/gitea/sessions
|
||||||
|
PROVIDER = file
|
||||||
|
|
||||||
|
[picture]
|
||||||
|
AVATAR_UPLOAD_PATH = /data/gitea/avatars
|
||||||
|
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
|
||||||
|
|
||||||
|
[attachment]
|
||||||
|
PATH = /data/gitea/attachments
|
||||||
|
|
||||||
|
[log]
|
||||||
|
MODE = console
|
||||||
|
LEVEL = info
|
||||||
|
ROOT_PATH = /data/gitea/log
|
||||||
|
|
||||||
|
[security]
|
||||||
|
INSTALL_LOCK = true
|
||||||
|
SECRET_KEY =
|
||||||
|
INTERNAL_TOKEN =
|
||||||
|
|
||||||
|
[service]
|
||||||
|
DISABLE_REGISTRATION = false
|
||||||
|
REQUIRE_SIGNIN_VIEW = false
|
||||||
|
REGISTER_EMAIL_CONFIRM = false
|
||||||
|
ENABLE_NOTIFY_MAIL = false
|
||||||
|
ALLOW_ONLY_EXTERNAL_REGISTRATION = false
|
||||||
|
ENABLE_CAPTCHA = false
|
||||||
|
DEFAULT_KEEP_EMAIL_PRIVATE = false
|
||||||
|
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
|
||||||
|
DEFAULT_ENABLE_TIMETRACKING = true
|
||||||
|
NO_REPLY_ADDRESS = noreply.example.org
|
||||||
|
|
||||||
|
[oauth2]
|
||||||
|
JWT_SECRET =
|
||||||
|
|
||||||
|
[mailer]
|
||||||
|
ENABLED = false
|
||||||
|
|
||||||
|
[openid]
|
||||||
|
ENABLE_OPENID_SIGNIN = true
|
||||||
|
ENABLE_OPENID_SIGNUP = true
|
||||||
37
ansible/templates/certbot-renew.j2
Normal file
37
ansible/templates/certbot-renew.j2
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Certbot renewal script for Forgejo SSL certificates
|
||||||
|
# Automatically created by Ansible
|
||||||
|
|
||||||
|
# Set up logging
|
||||||
|
LOG_FILE="/var/log/certbot-renewal.log"
|
||||||
|
echo "$(date): Starting certificate renewal" >> $LOG_FILE
|
||||||
|
|
||||||
|
# Stop Nginx to free up port 80
|
||||||
|
echo "$(date): Stopping Nginx" >> $LOG_FILE
|
||||||
|
systemctl stop nginx
|
||||||
|
if [ $? -ne 0 ]; then
|
||||||
|
echo "$(date): Failed to stop Nginx" >> $LOG_FILE
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Run Certbot renewal
|
||||||
|
echo "$(date): Running Certbot renewal" >> $LOG_FILE
|
||||||
|
docker run --rm -p 80:80 -p 443:443 \
|
||||||
|
-v /etc/letsencrypt:/etc/letsencrypt \
|
||||||
|
-v /var/lib/letsencrypt:/var/lib/letsencrypt \
|
||||||
|
certbot/certbot renew --standalone --non-interactive
|
||||||
|
|
||||||
|
if [ $? -ne 0 ]; then
|
||||||
|
echo "$(date): Certbot renewal failed" >> $LOG_FILE
|
||||||
|
# Even if renewal fails, we should restart Nginx
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Start Nginx again
|
||||||
|
echo "$(date): Starting Nginx" >> $LOG_FILE
|
||||||
|
systemctl start nginx
|
||||||
|
if [ $? -ne 0 ]; then
|
||||||
|
echo "$(date): Failed to start Nginx" >> $LOG_FILE
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "$(date): Certificate renewal completed successfully" >> $LOG_FILE
|
||||||
21
ansible/templates/docker-compose.yml.j2
Normal file
21
ansible/templates/docker-compose.yml.j2
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
networks:
|
||||||
|
forgejo:
|
||||||
|
external: false
|
||||||
|
|
||||||
|
services:
|
||||||
|
server:
|
||||||
|
image: codeberg.org/forgejo/forgejo:10
|
||||||
|
container_name: forgejo
|
||||||
|
environment:
|
||||||
|
- USER_UID={{ forgejo_user_uid }}
|
||||||
|
- USER_GID={{ forgejo_user_gid }}
|
||||||
|
restart: always
|
||||||
|
networks:
|
||||||
|
- forgejo
|
||||||
|
volumes:
|
||||||
|
- {{ forgejo_data_dir }}/data:/data
|
||||||
|
- /etc/timezone:/etc/timezone:ro
|
||||||
|
- /etc/localtime:/etc/localtime:ro
|
||||||
|
ports:
|
||||||
|
- '{{ forgejo_http_port }}:3000'
|
||||||
|
- '{{ forgejo_ssh_port }}:22'
|
||||||
12
ansible/templates/nginx-forgejo.conf.j2
Normal file
12
ansible/templates/nginx-forgejo.conf.j2
Normal file
|
|
@ -0,0 +1,12 @@
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name {{ forgejo_domain }};
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://localhost:{{ forgejo_http_port }}/;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
}
|
||||||
32
ansible/templates/nginx.conf.j2
Normal file
32
ansible/templates/nginx.conf.j2
Normal file
|
|
@ -0,0 +1,32 @@
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
server_name {{ forgejo_domain }};
|
||||||
|
|
||||||
|
# Redirect all HTTP requests to HTTPS
|
||||||
|
location / {
|
||||||
|
return 301 https://$host$request_uri;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 443 ssl;
|
||||||
|
server_name {{ forgejo_domain }};
|
||||||
|
|
||||||
|
ssl_certificate /etc/letsencrypt/live/{{ forgejo_domain }}/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/{{ forgejo_domain }}/privkey.pem;
|
||||||
|
|
||||||
|
# SSL configuration
|
||||||
|
ssl_protocols TLSv1.2 TLSv1.3;
|
||||||
|
ssl_prefer_server_ciphers on;
|
||||||
|
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
|
||||||
|
ssl_session_cache shared:SSL:10m;
|
||||||
|
ssl_session_timeout 10m;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://localhost:{{ forgejo_http_port }}/;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
}
|
||||||
|
}
|
||||||
44
cloud-init/cloud-init.sh
Normal file
44
cloud-init/cloud-init.sh
Normal file
|
|
@ -0,0 +1,44 @@
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Update and install dependencies
|
||||||
|
apt-get update
|
||||||
|
apt-get install -y \
|
||||||
|
apt-transport-https \
|
||||||
|
ca-certificates \
|
||||||
|
curl \
|
||||||
|
gnupg \
|
||||||
|
lsb-release \
|
||||||
|
python3 \
|
||||||
|
python3-pip \
|
||||||
|
git \
|
||||||
|
unzip
|
||||||
|
|
||||||
|
# Install Docker
|
||||||
|
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
|
||||||
|
echo \
|
||||||
|
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
|
||||||
|
$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
|
||||||
|
apt-get update
|
||||||
|
apt-get install -y docker-ce docker-ce-cli containerd.io
|
||||||
|
|
||||||
|
# Install Docker Compose
|
||||||
|
curl -L "https://github.com/docker/compose/releases/download/v2.20.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
|
||||||
|
chmod +x /usr/local/bin/docker-compose
|
||||||
|
|
||||||
|
# Add debian user to docker group
|
||||||
|
usermod -aG docker debian
|
||||||
|
|
||||||
|
# Install Ansible
|
||||||
|
pip3 install ansible
|
||||||
|
|
||||||
|
# Create directory for Ansible playbooks
|
||||||
|
mkdir -p /opt/ansible
|
||||||
|
chown -R debian:debian /opt/ansible
|
||||||
|
|
||||||
|
# Create directory for Forgejo
|
||||||
|
mkdir -p /opt/forgejo
|
||||||
|
chown -R debian:debian /opt/forgejo
|
||||||
|
|
||||||
|
# Signal cloud-init completion
|
||||||
|
touch /var/lib/cloud/instance/boot-finished
|
||||||
61
deploy.sh
Executable file
61
deploy.sh
Executable file
|
|
@ -0,0 +1,61 @@
|
||||||
|
#!/bin/bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# Colors for output
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
RED='\033[0;31m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
echo -e "${YELLOW}Starting deployment of GCP Data Engineering VM with Forgejo...${NC}"
|
||||||
|
|
||||||
|
# Check if terraform is installed
|
||||||
|
if ! command -v terraform &> /dev/null; then
|
||||||
|
echo -e "${RED}Terraform is not installed. Please install it first.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if ansible is installed
|
||||||
|
if ! command -v ansible &> /dev/null; then
|
||||||
|
echo -e "${RED}Ansible is not installed. Please install it first.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if terraform.tfvars exists
|
||||||
|
if [ ! -f terraform/terraform.tfvars ]; then
|
||||||
|
echo -e "${YELLOW}terraform.tfvars not found. Creating from example file...${NC}"
|
||||||
|
cp terraform/terraform.tfvars.example terraform/terraform.tfvars
|
||||||
|
echo -e "${RED}Please edit terraform/terraform.tfvars with your GCP project details before continuing.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Initialize Terraform
|
||||||
|
echo -e "${GREEN}Initializing Terraform...${NC}"
|
||||||
|
cd terraform
|
||||||
|
terraform init
|
||||||
|
|
||||||
|
# Apply Terraform configuration
|
||||||
|
echo -e "${GREEN}Applying Terraform configuration...${NC}"
|
||||||
|
terraform apply -auto-approve
|
||||||
|
|
||||||
|
# Get the VM IP address
|
||||||
|
VM_IP=$(terraform output -raw instance_ip)
|
||||||
|
echo -e "${GREEN}VM created with IP: ${VM_IP}${NC}"
|
||||||
|
cd ..
|
||||||
|
|
||||||
|
# Update Ansible inventory with VM IP
|
||||||
|
echo -e "${GREEN}Updating Ansible inventory with VM IP...${NC}"
|
||||||
|
sed -i "s/ansible_host: \"{{ vm_ip_address }}\"/ansible_host: \"${VM_IP}\"/g" ansible/inventory.yml
|
||||||
|
|
||||||
|
# Wait for VM to be ready
|
||||||
|
echo -e "${YELLOW}Waiting for VM to be ready (60 seconds)...${NC}"
|
||||||
|
sleep 60
|
||||||
|
|
||||||
|
# Run Ansible playbook
|
||||||
|
echo -e "${GREEN}Running Ansible playbook to configure Forgejo...${NC}"
|
||||||
|
cd ansible
|
||||||
|
ansible-playbook -i inventory.yml forgejo.yml
|
||||||
|
|
||||||
|
echo -e "${GREEN}Deployment completed successfully!${NC}"
|
||||||
|
echo -e "${YELLOW}Forgejo should be accessible at https://your-duckdns-subdomain.duckdns.org${NC}"
|
||||||
|
echo -e "${YELLOW}Please allow a few minutes for DNS propagation and SSL certificate setup.${NC}"
|
||||||
66
terraform/main.tf
Normal file
66
terraform/main.tf
Normal file
|
|
@ -0,0 +1,66 @@
|
||||||
|
terraform {
|
||||||
|
required_providers {
|
||||||
|
google = {
|
||||||
|
source = "hashicorp/google"
|
||||||
|
version = "~> 4.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
backend "local" {
|
||||||
|
path = "terraform.tfstate"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
provider "google" {
|
||||||
|
project = var.project_id
|
||||||
|
region = var.region
|
||||||
|
zone = var.zone
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "google_compute_instance" "data_engineering_vm" {
|
||||||
|
name = var.instance_name
|
||||||
|
machine_type = var.machine_type
|
||||||
|
zone = var.zone
|
||||||
|
|
||||||
|
boot_disk {
|
||||||
|
initialize_params {
|
||||||
|
image = "debian-cloud/debian-11"
|
||||||
|
size = var.disk_size_gb
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
network_interface {
|
||||||
|
network = "default"
|
||||||
|
access_config {
|
||||||
|
// Ephemeral public IP
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata = {
|
||||||
|
ssh-keys = "${var.ssh_username}:${file(var.ssh_pub_key_path)}"
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata_startup_script = file("${path.module}/../cloud-init/cloud-init.sh")
|
||||||
|
|
||||||
|
tags = ["http-server", "https-server", "ssh"]
|
||||||
|
|
||||||
|
service_account {
|
||||||
|
scopes = ["cloud-platform"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
resource "google_compute_firewall" "forgejo" {
|
||||||
|
name = "allow-forgejo"
|
||||||
|
network = "default"
|
||||||
|
|
||||||
|
allow {
|
||||||
|
protocol = "tcp"
|
||||||
|
ports = ["22", "80", "443", "3000", "222"]
|
||||||
|
}
|
||||||
|
|
||||||
|
source_ranges = ["0.0.0.0/0"]
|
||||||
|
target_tags = ["http-server", "https-server", "ssh"]
|
||||||
|
}
|
||||||
|
|
||||||
|
output "instance_ip" {
|
||||||
|
value = google_compute_instance.data_engineering_vm.network_interface[0].access_config[0].nat_ip
|
||||||
|
}
|
||||||
8
terraform/terraform.tfvars
Normal file
8
terraform/terraform.tfvars
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
project_id = "homelab-454516"
|
||||||
|
region = "us-central1"
|
||||||
|
zone = "us-central1-a"
|
||||||
|
instance_name = "data-engineering-vm"
|
||||||
|
machine_type = "e2-medium"
|
||||||
|
disk_size_gb = 50
|
||||||
|
ssh_username = "debian"
|
||||||
|
ssh_pub_key_path = "~/.ssh/id_rsa.pub"
|
||||||
8
terraform/terraform.tfvars.example
Normal file
8
terraform/terraform.tfvars.example
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
project_id = "your-gcp-project-id"
|
||||||
|
region = "us-central1"
|
||||||
|
zone = "us-central1-a"
|
||||||
|
instance_name = "data-engineering-vm"
|
||||||
|
machine_type = "e2-medium"
|
||||||
|
disk_size_gb = 50
|
||||||
|
ssh_username = "debian"
|
||||||
|
ssh_pub_key_path = "~/.ssh/id_rsa.pub"
|
||||||
46
terraform/variables.tf
Normal file
46
terraform/variables.tf
Normal file
|
|
@ -0,0 +1,46 @@
|
||||||
|
variable "project_id" {
|
||||||
|
description = "The GCP project ID"
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "region" {
|
||||||
|
description = "The GCP region"
|
||||||
|
type = string
|
||||||
|
default = "us-central1"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "zone" {
|
||||||
|
description = "The GCP zone"
|
||||||
|
type = string
|
||||||
|
default = "us-central1-a"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "instance_name" {
|
||||||
|
description = "Name of the VM instance"
|
||||||
|
type = string
|
||||||
|
default = "data-engineering-vm"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "machine_type" {
|
||||||
|
description = "The machine type for the VM"
|
||||||
|
type = string
|
||||||
|
default = "e2-medium"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "disk_size_gb" {
|
||||||
|
description = "Boot disk size in GB"
|
||||||
|
type = number
|
||||||
|
default = 50
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "ssh_username" {
|
||||||
|
description = "SSH username for the VM"
|
||||||
|
type = string
|
||||||
|
default = "debian"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "ssh_pub_key_path" {
|
||||||
|
description = "Path to the SSH public key file"
|
||||||
|
type = string
|
||||||
|
default = "~/.ssh/id_rsa.pub"
|
||||||
|
}
|
||||||
Loading…
Add table
Reference in a new issue