From 9f2b34025e52e7b493a55675e732e967ddc192fb Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Thu, 14 Jul 2022 10:21:33 +0200 Subject: [PATCH 01/22] Draft --- docs/hosting/server-setups/google-cloud.md | 151 +++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/hosting/server-setups/google-cloud.md diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md new file mode 100644 index 000000000..72f31924c --- /dev/null +++ b/docs/hosting/server-setups/google-cloud.md @@ -0,0 +1,151 @@ +# Hosting n8n on Google Cloud + +This hosting guide shows you how to self-host n8n on Google Cloud. It uses n8n and Caddy as a reverse proxy using Docker Compose to create the necessary resources. + +## Hosting options + +Google Cloud offers several ways suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes (containers running with Kubernetes). + +This guide uses compute engine as it allows for persistent data by default, however, other options could also work for self-hosting n8n. + +You can use one of the marketplace Docker images, or install Docker and Docker Compose yourself. If you would rather create an instance using the gcloud command line tool, find more details in the [Google Cloud Platform documentation](https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). + +## Login to instance + +The remainder of the steps in this guide require you to login to the instance, GCP offers a variety of ways to do this, and you can find connection commands from the _SSH_ dropdown of the instance details page. + +## Prerequisites + +### Create folders + +Both n8n and Caddy require creating folders and files that the host operating system copies to Docker containers. Create the following on the instance in a location accessible by Docker: + +- _caddy_config_ +- _caddy_config/Caddyfile_ +- _caddy_data_ +- _local_files_ + +### Setup DNS + +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. + +If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". + + + +### Open ports + +To set up secure connections to the instance, you need to open Firewall rules. You can do this when creating or editing an instance from the _Firewall_ section, or you can [use the gcloud CLI tool to open the ports](https://cloud.google.com/vpc/docs/firewall-rules-logging). + +## Configure n8n + +Create an _.env_ file in the same folder you will run Docker Compose from, and add the following, replacing the values with your own: + +```env +# Path where you created folders and files earlier +DATA_FOLDER=/root/n8n + +# The top level domain to serve from +DOMAIN_NAME=example.com + +# The subdomain to serve from +SUBDOMAIN=n8n + +# DOMAIN_NAME and SUBDOMAIN combined decide where n8n will be reachable from +# above example would result in: https://n8n.example.com + +# The user name to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_USER=user + +# The password to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_PASSWORD=password + +# Optional timezone to set which gets used by Cron-Node by default +# If not set New York time will be used +GENERIC_TIMEZONE=Europe/Berlin + +# The email address to use for the SSL certificate creation +SSL_EMAIL=example@example.com +``` + +## Create Docker Compose file + +Create a _docker-compose.yml_ file, and add the following: + +```yaml +version: "3.7" + +services: + caddy: + image: caddy:latest + restart: unless-stopped + ports: + - "80:80" + - "443:443" + volumes: + - ${DATA_FOLDER}/caddy_data:/data + - ${DATA_FOLDER}/caddy_config:/config + - "${DATA_FOLDER}/caddy_config/Caddyfile:/etc/caddy/Caddyfile" + + n8n: + image: n8nio/n8n + restart: always + ports: + - 5678:5678 + environment: + - N8N_BASIC_AUTH_ACTIVE=true + - N8N_BASIC_AUTH_USER + - N8N_BASIC_AUTH_PASSWORD + - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME} + - N8N_PORT=5678 + - N8N_PROTOCOL=https + - NODE_ENV=production + - WEBHOOK_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/ + - GENERIC_TIMEZONE=${GENERIC_TIMEZONE} + volumes: + - ${DATA_FOLDER}/.n8n:/home/node/.n8n + - ${DATA_FOLDER}/local_files:/files + +volumes: + caddy_data: + external: true + caddy_config: +``` + +## Configure Caddy + +Open the _Caddyfile_ you created earlier, and add the following configuration, adding your subdomain: + +```text + { + reverse_proxy n8n:5678 { + flush_interval -1 + } +} +``` + +Using `n8n` in the configuration + +## Create Docker volume + +To persist Caddy configuration between restarts, the Docker Compose file above specifies an `external` volume, create that with the following command: + +```shell +docker volume create caddy_data +``` + +## Start Docker Compose + +Start n8n and Caddy with the following command: + +```shell +docker-compose up -d +``` + +Open the URL formed of the subdomain and domain name defined earlier, enter the user name and password defined earlier, and you should be able to access n8n. + +Stop n8n and Caddy with the following command: + +```shell +sudo docker-compose stop +``` \ No newline at end of file From 23f9534f4b48e0c6bb3babfc11a07ab9dfb229c0 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Thu, 14 Jul 2022 10:21:33 +0200 Subject: [PATCH 02/22] Draft --- docs/hosting/server-setups/google-cloud.md | 151 +++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/hosting/server-setups/google-cloud.md diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md new file mode 100644 index 000000000..72f31924c --- /dev/null +++ b/docs/hosting/server-setups/google-cloud.md @@ -0,0 +1,151 @@ +# Hosting n8n on Google Cloud + +This hosting guide shows you how to self-host n8n on Google Cloud. It uses n8n and Caddy as a reverse proxy using Docker Compose to create the necessary resources. + +## Hosting options + +Google Cloud offers several ways suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes (containers running with Kubernetes). + +This guide uses compute engine as it allows for persistent data by default, however, other options could also work for self-hosting n8n. + +You can use one of the marketplace Docker images, or install Docker and Docker Compose yourself. If you would rather create an instance using the gcloud command line tool, find more details in the [Google Cloud Platform documentation](https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). + +## Login to instance + +The remainder of the steps in this guide require you to login to the instance, GCP offers a variety of ways to do this, and you can find connection commands from the _SSH_ dropdown of the instance details page. + +## Prerequisites + +### Create folders + +Both n8n and Caddy require creating folders and files that the host operating system copies to Docker containers. Create the following on the instance in a location accessible by Docker: + +- _caddy_config_ +- _caddy_config/Caddyfile_ +- _caddy_data_ +- _local_files_ + +### Setup DNS + +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. + +If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". + + + +### Open ports + +To set up secure connections to the instance, you need to open Firewall rules. You can do this when creating or editing an instance from the _Firewall_ section, or you can [use the gcloud CLI tool to open the ports](https://cloud.google.com/vpc/docs/firewall-rules-logging). + +## Configure n8n + +Create an _.env_ file in the same folder you will run Docker Compose from, and add the following, replacing the values with your own: + +```env +# Path where you created folders and files earlier +DATA_FOLDER=/root/n8n + +# The top level domain to serve from +DOMAIN_NAME=example.com + +# The subdomain to serve from +SUBDOMAIN=n8n + +# DOMAIN_NAME and SUBDOMAIN combined decide where n8n will be reachable from +# above example would result in: https://n8n.example.com + +# The user name to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_USER=user + +# The password to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_PASSWORD=password + +# Optional timezone to set which gets used by Cron-Node by default +# If not set New York time will be used +GENERIC_TIMEZONE=Europe/Berlin + +# The email address to use for the SSL certificate creation +SSL_EMAIL=example@example.com +``` + +## Create Docker Compose file + +Create a _docker-compose.yml_ file, and add the following: + +```yaml +version: "3.7" + +services: + caddy: + image: caddy:latest + restart: unless-stopped + ports: + - "80:80" + - "443:443" + volumes: + - ${DATA_FOLDER}/caddy_data:/data + - ${DATA_FOLDER}/caddy_config:/config + - "${DATA_FOLDER}/caddy_config/Caddyfile:/etc/caddy/Caddyfile" + + n8n: + image: n8nio/n8n + restart: always + ports: + - 5678:5678 + environment: + - N8N_BASIC_AUTH_ACTIVE=true + - N8N_BASIC_AUTH_USER + - N8N_BASIC_AUTH_PASSWORD + - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME} + - N8N_PORT=5678 + - N8N_PROTOCOL=https + - NODE_ENV=production + - WEBHOOK_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/ + - GENERIC_TIMEZONE=${GENERIC_TIMEZONE} + volumes: + - ${DATA_FOLDER}/.n8n:/home/node/.n8n + - ${DATA_FOLDER}/local_files:/files + +volumes: + caddy_data: + external: true + caddy_config: +``` + +## Configure Caddy + +Open the _Caddyfile_ you created earlier, and add the following configuration, adding your subdomain: + +```text + { + reverse_proxy n8n:5678 { + flush_interval -1 + } +} +``` + +Using `n8n` in the configuration + +## Create Docker volume + +To persist Caddy configuration between restarts, the Docker Compose file above specifies an `external` volume, create that with the following command: + +```shell +docker volume create caddy_data +``` + +## Start Docker Compose + +Start n8n and Caddy with the following command: + +```shell +docker-compose up -d +``` + +Open the URL formed of the subdomain and domain name defined earlier, enter the user name and password defined earlier, and you should be able to access n8n. + +Stop n8n and Caddy with the following command: + +```shell +sudo docker-compose stop +``` \ No newline at end of file From 88688d321f02d4e7c17cf91628ff689751f9a5db Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 8 Aug 2022 12:42:06 +0200 Subject: [PATCH 03/22] Draft --- docs/hosting/server-setups/google-cloud.md | 32 +++- docs/hosting/server-setups/temp.md | 178 +++++++++++++++++++++ 2 files changed, 204 insertions(+), 6 deletions(-) create mode 100644 docs/hosting/server-setups/temp.md diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 72f31924c..461e4f604 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -1,20 +1,40 @@ # Hosting n8n on Google Cloud -This hosting guide shows you how to self-host n8n on Google Cloud. It uses n8n and Caddy as a reverse proxy using Docker Compose to create the necessary resources. +This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses n8n and Caddy as a reverse proxy using Kubernetes to create the necessary resources. + +## Prerequisites + ## Hosting options -Google Cloud offers several ways suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes (containers running with Kubernetes). +Google Cloud offers several ways suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes Engine (containers running with Kubernetes). + +This guide uses the Google Kubernetes Engine (GKE) as the hosting option. Using Kubernetes requires some additional complexity and configuration, but is the best method for scaling n8n as demand changes. + +The steps in this guide use the Google Cloud UI, but you can also use the [gcloud command line tool](https://cloud.google.com/sdk/gcloud/) instead. + +## Create project + +GCP encourages you to create projects to logically organize resources and configuration. Create a new project for your n8n deployment by clicking the project dropdown menu and then the _NEW PROJECT_ button. Then select the newly created project and as you follow other steps in this guide, make sure you have the correct project selected. + + + +## Enable the Kubernetes Engine API + +GKE isn't enabled by default, search for "Kubernetes" in the top search bar and select "Kubernetes Engine" from the results. + +Enable the Kubernetes Engine API by clicking the __Enable__ button. + +## Create a cluster + +From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. -This guide uses compute engine as it allows for persistent data by default, however, other options could also work for self-hosting n8n. -You can use one of the marketplace Docker images, or install Docker and Docker Compose yourself. If you would rather create an instance using the gcloud command line tool, find more details in the [Google Cloud Platform documentation](https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). ## Login to instance -The remainder of the steps in this guide require you to login to the instance, GCP offers a variety of ways to do this, and you can find connection commands from the _SSH_ dropdown of the instance details page. +The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string that needs you to have [the gcloud command line tool installed](https://cloud.google.com/sdk/gcloud/). Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. -## Prerequisites ### Create folders diff --git a/docs/hosting/server-setups/temp.md b/docs/hosting/server-setups/temp.md new file mode 100644 index 000000000..61d58061e --- /dev/null +++ b/docs/hosting/server-setups/temp.md @@ -0,0 +1,178 @@ +# Hosting n8n on DigitalOcean + +This hosting guide shows you how to self-host n8n on a DigitalOcean droplet. It uses n8n and [Caddy](http://caddyserver.com){:target="_blank" class=.external-link} (a reverse proxy) to allow access to the Droplet from the internet . It uses [Docker Compose](https://docs.docker.com/compose/){:target="_blank"} to create and define the application components and how they work together. + +## Create a Droplet + +Log in to DigitalOcean and select the project to host the Droplet and click **Droplets** from the **Manage** menu. You can use [the Docker image](https://marketplace.digitalocean.com/apps/docker){:target="_blank" class=.external-link} from the **Marketplace** tab or use one of the Linux distributions and [install Docker yourself](https://www.docker.com/get-started/){:target="_blank" class=.external-link}. The rest of this tutorial assumes you have Docker installed on the Droplet. + +## Log in to your Droplet + +The remainder of the steps in this guide require you to log in to the Droplet using a terminal with SSH. [Find more details on how to do this in the Digital Ocean documentation](https://docs.digitalocean.com/products/droplets/how-to/connect-with-ssh/){:target="_blank"}. + +## Create folders and files + +Both n8n and Caddy require creating folders that the host operating system (the DigitalOcean Droplet) copies to Docker containers to make them available to Docker. + +Create the following on the Droplet in a location accessible by Docker. If you run Docker as root user, you can create them anywhere. If you run Docker "[Rootless](https://docs.docker.com/engine/security/rootless/){:target="_blank"}" for better security, create them in a location that user has access to: + +- `caddy_config`: Holds the Caddy configuration files. +- `caddy_data`: A cache folder for Caddy. +- `local_files`: A folder for files you upload or add via n8n. + +Use the following command to create the folders: + +```shell +mkdir caddy_config caddy_data local_files +``` + +### Create Docker volume + +To persist the Caddy cache between restarts and speed up start times, create [a Docker volume](https://docs.docker.com/storage/volumes/){:target="_blank"} that Docker reuses between restarts. + +```shell +docker volume create caddy_data +``` + +## Set up DNS + +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the Droplet. The exact steps for this depend on your DNS provider, but typically you need to create a new [CNAME or A record](https://ns1.com/resources/dns-records-explained){:target="_blank"} for the n8n subdomain. + +## Open ports + +n8n runs as a web application, so the Droplet needs to allow incoming access to traffic on port 80 for non-secure traffic, and port 443 for secure traffic. + +Open the following ports in the Droplet's firewall by running the following two commands: + +```shell +sudo ufw allow 80 +sudo ufw allow 443 +``` + +## Configure n8n + +Create an `.env` file in the same folder you will run Docker Compose from, and add the following, replacing the values with your own: + +Create the file: + +```shell +nano .env +``` + +Add the contents to the file: + +```env +# Path where you created folders earlier +DATA_FOLDER=/root/n8n + +# The top level domain to serve from, this should be the same as the subdomain you created above +DOMAIN_NAME=example.com + +# The subdomain to serve from +SUBDOMAIN=n8n + +# DOMAIN_NAME and SUBDOMAIN combined decide where n8n will be reachable from +# above example would result in: https://n8n.example.com + +# The user name to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_USER=user + +# The password to use for authentication - IMPORTANT ALWAYS CHANGE! +N8N_BASIC_AUTH_PASSWORD=password + +# Optional timezone to set which gets used by Cron-Node by default +# If not set New York time will be used +GENERIC_TIMEZONE=Europe/Berlin + +# The email address to use for the SSL certificate creation +SSL_EMAIL=example@example.com +``` + +## Create Docker Compose file + +The Docker compose file defines the services the application needs, in this case Caddy and n8n. + +- The Caddy service definition defines the ports it uses and the local volumes to copy to the containers. +- The n8n service definition defines the ports it uses, the environment variables n8n needs to run (some defined in the `.env` file), and the volumes it needs to copy to the containers. + +Create a `docker-compose.yml` file, make sure to create it in the same location as the `.env` file: + +```shell +nano docker-compose.yml +``` + +Add the following to the file: + +```yaml +version: "3.7" + +services: + caddy: + image: caddy:latest + restart: unless-stopped + ports: + - "80:80" + - "443:443" + volumes: + - ${DATA_FOLDER}/caddy_data:/data + - ${DATA_FOLDER}/caddy_config:/config + - ${DATA_FOLDER}/caddy_config/Caddyfile:/etc/caddy/Caddyfile + + n8n: + image: n8nio/n8n + restart: always + ports: + - 5678:5678 + environment: + - N8N_BASIC_AUTH_ACTIVE=true + - N8N_BASIC_AUTH_USER + - N8N_BASIC_AUTH_PASSWORD + - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME} + - N8N_PORT=5678 + - N8N_PROTOCOL=https + - NODE_ENV=production + - WEBHOOK_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/ + - GENERIC_TIMEZONE=${GENERIC_TIMEZONE} + volumes: + - ${DATA_FOLDER}/.n8n:/home/node/.n8n + - ${DATA_FOLDER}/local_files:/files + +volumes: + caddy_data: + external: true + caddy_config: +``` + +## Configure Caddy + +Caddy needs to know which domains it should serve, and which port to expose to the outside world. Create a `Caddyfile` file in the *caddy_config* folder. + +```shell +nano caddy_config/Caddyfile +``` + +Add the following configuration, adding your subdomain. The `n8n` tells Caddy to use the service definition defined in the `docker-compose.yml` file: + +```text + { + reverse_proxy n8n:5678 { + flush_interval -1 + } +} +``` + +## Start Docker Compose + +Start n8n and Caddy with the following command: + +```shell +docker-compose up -d +``` + +Open the URL formed of the subdomain and domain name defined earlier, enter the user name and password defined earlier, and you should be able to access n8n. + +You can stop n8n and Caddy with the following command: + +```shell +sudo docker-compose stop +``` \ No newline at end of file From fcf02b4ea9c71df0959e3436b31231f3ffb152ee Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 15 Aug 2022 13:54:03 +0200 Subject: [PATCH 04/22] Draft --- docs/hosting/server-setups/google-cloud.md | 26 ++++++++++++++-------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 461e4f604..c17326497 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -36,6 +36,23 @@ From the GKE service page, click the **Clusters** menu item and then the **CREAT The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string that needs you to have [the gcloud command line tool installed](https://cloud.google.com/sdk/gcloud/). Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. + + +## Setup DNS + +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. + +If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". + + +## Open ports + +To set up secure connections to the instance, you need to open Firewall rules. You can do this when creating or editing an instance from the _Firewall_ section, or you can [use the gcloud CLI tool to open the ports](https://cloud.google.com/vpc/docs/firewall-rules-logging). + + + + + ### Create folders Both n8n and Caddy require creating folders and files that the host operating system copies to Docker containers. Create the following on the instance in a location accessible by Docker: @@ -45,17 +62,8 @@ Both n8n and Caddy require creating folders and files that the host operating sy - _caddy_data_ - _local_files_ -### Setup DNS -n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. -If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". - - - -### Open ports - -To set up secure connections to the instance, you need to open Firewall rules. You can do this when creating or editing an instance from the _Firewall_ section, or you can [use the gcloud CLI tool to open the ports](https://cloud.google.com/vpc/docs/firewall-rules-logging). ## Configure n8n From 4cbef811024d81195d782710a64bac9e24fc65ee Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Thu, 18 Aug 2022 15:55:13 +0200 Subject: [PATCH 05/22] Draft --- docs/hosting/server-setups/google-cloud.md | 49 ++++++++++++---------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index c17326497..43c66f97d 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -1,9 +1,9 @@ # Hosting n8n on Google Cloud -This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses n8n and Caddy as a reverse proxy using Kubernetes to create the necessary resources. +This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses n8n with Postgres as a database backend using Kubernetes to manage the necessary resources and reverse proxy. ## Prerequisites - +- [The gcloud command line tool](https://cloud.google.com/sdk/gcloud/) ## Hosting options @@ -11,7 +11,7 @@ Google Cloud offers several ways suitable for hosting n8n, including Cloud Run ( This guide uses the Google Kubernetes Engine (GKE) as the hosting option. Using Kubernetes requires some additional complexity and configuration, but is the best method for scaling n8n as demand changes. -The steps in this guide use the Google Cloud UI, but you can also use the [gcloud command line tool](https://cloud.google.com/sdk/gcloud/) instead. +Most of the steps in this guide use the Google Cloud UI, but you can also use the [gcloud command line tool](https://cloud.google.com/sdk/gcloud/) instead to undertake all the steps. ## Create project @@ -27,13 +27,13 @@ Enable the Kubernetes Engine API by clicking the __Enable__ button. ## Create a cluster -From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. +From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. ## Login to instance -The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string that needs you to have [the gcloud command line tool installed](https://cloud.google.com/sdk/gcloud/). Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. +The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. @@ -44,29 +44,36 @@ n8n typically operates on a subdomain. Create a DNS record with your provider fo If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". + + +## Clone repo +The next few steps walk through the important parts of what the manifests configure. ## Configure n8n +### Create a volume for file storage + +During initial setup and certain workflows, n8n needs to write files to disk and needs a persistent volume to do so. + +The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. + +```yaml +… +volumes: + - name: n8n-claim0 + persistentVolumeClaim: + claimName: n8n-claim0 +… +``` + + +## Environment variables + Create an _.env_ file in the same folder you will run Docker Compose from, and add the following, replacing the values with your own: ```env From ad7b81d1221b4186caad621829f380711c6f8bb0 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Tue, 23 Aug 2022 15:48:13 +0200 Subject: [PATCH 06/22] GCP draft --- docs/hosting/server-setups/google-cloud.md | 160 +++++++-------------- mkdocs.yml | 1 + 2 files changed, 55 insertions(+), 106 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 43c66f97d..5c7c4fbf6 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -3,7 +3,8 @@ This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses n8n with Postgres as a database backend using Kubernetes to manage the necessary resources and reverse proxy. ## Prerequisites -- [The gcloud command line tool](https://cloud.google.com/sdk/gcloud/) + +- [The gcloud command line tool](https://cloud.google.com/sdk/gcloud/){:target="_blank" .external-link} ## Hosting options @@ -11,33 +12,26 @@ Google Cloud offers several ways suitable for hosting n8n, including Cloud Run ( This guide uses the Google Kubernetes Engine (GKE) as the hosting option. Using Kubernetes requires some additional complexity and configuration, but is the best method for scaling n8n as demand changes. -Most of the steps in this guide use the Google Cloud UI, but you can also use the [gcloud command line tool](https://cloud.google.com/sdk/gcloud/) instead to undertake all the steps. +Most of the steps in this guide use the Google Cloud UI, but you can also use the [gcloud command line tool](https://cloud.google.com/sdk/gcloud/){:target="_blank" .external-link} instead to undertake all the steps. ## Create project GCP encourages you to create projects to logically organize resources and configuration. Create a new project for your n8n deployment by clicking the project dropdown menu and then the _NEW PROJECT_ button. Then select the newly created project and as you follow other steps in this guide, make sure you have the correct project selected. - - ## Enable the Kubernetes Engine API GKE isn't enabled by default, search for "Kubernetes" in the top search bar and select "Kubernetes Engine" from the results. -Enable the Kubernetes Engine API by clicking the __Enable__ button. +Enable the Kubernetes Engine API by clicking the **Enable** button. ## Create a cluster From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. - - ## Login to instance The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. - - - ## Setup DNS n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. @@ -51,9 +45,34 @@ To set up http connections to the instance, you need to open Firewall rules. You ## Clone repo - The next few steps walk through the important parts of what the manifests configure. +## Configure Postgres + +For larger scale n8n deployments, Postgres provides a more robust database backend than SQLite. + +### Create a volume for persistent storage + +To maintain data between pod restarts, the Postgres deployment needs a persistent volume. Running Postgres on GCP requires a specific Kubernetes Storage Class, [you can read this guide for specifics](https://cloud.google.com/architecture/deploying-highly-available-postgresql-with-gke){:target="_blank" .external-link}, but the `storage.yaml` manifest creates it for you. You may want to change the regions to create the storage in under the `allowedTopologies` > `matchedLabelExpressions` > `values` key. By default, they're set to "us-central". + +```yaml +… +allowedTopologies: + - matchLabelExpressions: + - key: failure-domain.beta.kubernetes.io/zone + values: + - us-central1-b + - us-central1-c +``` + +### Environment variables + +Postgres needs some environment variables set to pass to the application running in the containers. + +The example `postgres-secret.yaml` file contains placeholders you need to replace with values of your own for user details and the database to use. + +The `postgres-deployment.yaml` manifest then uses the values from this manifest file to send to the application pods. + ## Configure n8n ### Create a volume for file storage @@ -71,116 +90,45 @@ volumes: … ``` +### Environment variables -## Environment variables +n8n needs some environment variables set to pass to the application running in the containers. -Create an _.env_ file in the same folder you will run Docker Compose from, and add the following, replacing the values with your own: +The example `n8n-secret.yaml` file contains placeholders you need to replace with values of your own for authentication details. -```env -# Path where you created folders and files earlier -DATA_FOLDER=/root/n8n +## Deployments -# The top level domain to serve from -DOMAIN_NAME=example.com +The two deployment manifests (`n8n-deployment.yaml` and `postgres-deployment.yaml`) define the n8n and Postgres applications to Kubernetes. -# The subdomain to serve from -SUBDOMAIN=n8n +The manifests define the following: -# DOMAIN_NAME and SUBDOMAIN combined decide where n8n will be reachable from -# above example would result in: https://n8n.example.com +- Send the environment variables defined to each application pod +- Define the container image to use +- Set resource consumption limits. This is left empty in the example manifests, but you should set them to something appropriate for your deployment. +- The `volumes` defined earlier and `volumeMounts` to define the path in the container to mount volumes. +- Scaling and restart policies. The example manifests define only one instance of each pod, you should change this to meet your needs. -# The user name to use for authentication - IMPORTANT ALWAYS CHANGE! -N8N_BASIC_AUTH_USER=user +## Services -# The password to use for authentication - IMPORTANT ALWAYS CHANGE! -N8N_BASIC_AUTH_PASSWORD=password +The two service manifests (`postgres-service.yaml` and `n8n-service.yaml`) expose the services to the outside world using the Kubernetes load balancer using ports 5432 and 5678 respectively by default. -# Optional timezone to set which gets used by Cron-Node by default -# If not set New York time will be used -GENERIC_TIMEZONE=Europe/Berlin +## Send to Kubernetes cluster -# The email address to use for the SSL certificate creation -SSL_EMAIL=example@example.com -``` - -## Create Docker Compose file - -Create a _docker-compose.yml_ file, and add the following: - -```yaml -version: "3.7" - -services: - caddy: - image: caddy:latest - restart: unless-stopped - ports: - - "80:80" - - "443:443" - volumes: - - ${DATA_FOLDER}/caddy_data:/data - - ${DATA_FOLDER}/caddy_config:/config - - "${DATA_FOLDER}/caddy_config/Caddyfile:/etc/caddy/Caddyfile" - - n8n: - image: n8nio/n8n - restart: always - ports: - - 5678:5678 - environment: - - N8N_BASIC_AUTH_ACTIVE=true - - N8N_BASIC_AUTH_USER - - N8N_BASIC_AUTH_PASSWORD - - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME} - - N8N_PORT=5678 - - N8N_PROTOCOL=https - - NODE_ENV=production - - WEBHOOK_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/ - - GENERIC_TIMEZONE=${GENERIC_TIMEZONE} - volumes: - - ${DATA_FOLDER}/.n8n:/home/node/.n8n - - ${DATA_FOLDER}/local_files:/files - -volumes: - caddy_data: - external: true - caddy_config: -``` - -## Configure Caddy - -Open the _Caddyfile_ you created earlier, and add the following configuration, adding your subdomain: - -```text - { - reverse_proxy n8n:5678 { - flush_interval -1 - } -} -``` - -Using `n8n` in the configuration - -## Create Docker volume - -To persist Caddy configuration between restarts, the Docker Compose file above specifies an `external` volume, create that with the following command: +Send all the manifests to the cluster with the following command: ```shell -docker volume create caddy_data +kubectl apply -f . ``` -## Start Docker Compose +!!! note "Namespace error" + You may see an error message about not finding an "n8n" namespace as that resources isn't ready yet. You can run the same command again, or apply the namespace manifest first with the following command: -Start n8n and Caddy with the following command: + ```shell + kubectl apply -f namespace.yaml + ``` + +Remove the resources created by the manifests with the following command: ```shell -docker-compose up -d +kubectl delete -f . ``` - -Open the URL formed of the subdomain and domain name defined earlier, enter the user name and password defined earlier, and you should be able to access n8n. - -Stop n8n and Caddy with the following command: - -```shell -sudo docker-compose stop -``` \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 97142d5df..03c674177 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -182,6 +182,7 @@ nav: - Docker Compose: hosting/server-setups/docker-compose.md - Caddy: hosting/server-setups/caddy.md - Digital Ocean: hosting/server-setups/digital-ocean.md + - Google Cloud: hosting/server-setups/google-cloud.md - Databases: - Overview: hosting/databases/index.md - Supported databases and settings: hosting/databases/supported-databases-settings.md From d3970b99d4493dd95edf651de6dc4dead3ce486e Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 5 Sep 2022 13:57:07 +0200 Subject: [PATCH 07/22] Clone steps --- docs/hosting/server-setups/google-cloud.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 5c7c4fbf6..dec14759d 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -43,9 +43,21 @@ If the instance doesn't already have a static IP address, you can assign one to To set up http connections to the instance, you need to open Firewall rules. You can do this when creating or editing an instance from the _Firewall_ section, or you can [use the gcloud CLI tool to open the ports](https://cloud.google.com/vpc/docs/firewall-rules-logging). --> -## Clone repo +## Clone configuration repository -The next few steps walk through the important parts of what the manifests configure. +Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/aws){:target=_blank .external-link} locally. The following steps will tell you which file configures what and what you need to change. + +Clone the repository with the following command: + +```shell +git clone https://github.com/n8n-io/n8n-kubernetes-hosting/tree/aws +``` + +And change directory to the root of the repository you cloned: + +```shell +cd aws +``` ## Configure Postgres From e8b61458545be874e8ed9a952dd7f6cce3e033c0 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 19 Sep 2022 13:04:35 +0200 Subject: [PATCH 08/22] Update --- docs/hosting/server-setups/google-cloud.md | 25 +++++----------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index dec14759d..63509415c 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -28,9 +28,9 @@ Enable the Kubernetes Engine API by clicking the **Enable** button. From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. -## Login to instance +## Set Kubectl context -The remainder of the steps in this guide require you to login to the instance via an SSH connection. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. +The remainder of the steps in this guide require you to set the GCP instance as the Kubectl context. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. ## Setup DNS @@ -45,18 +45,18 @@ To set up http connections to the instance, you need to open Firewall rules. You ## Clone configuration repository -Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/aws){:target=_blank .external-link} locally. The following steps will tell you which file configures what and what you need to change. +Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp){:target=_blank .external-link} locally. The following steps will tell you which file configures what and what you need to change. Clone the repository with the following command: ```shell -git clone https://github.com/n8n-io/n8n-kubernetes-hosting/tree/aws +git clone https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp ``` And change directory to the root of the repository you cloned: ```shell -cd aws +cd gcp ``` ## Configure Postgres @@ -87,21 +87,6 @@ The `postgres-deployment.yaml` manifest then uses the values from this manifest ## Configure n8n -### Create a volume for file storage - -During initial setup and certain workflows, n8n needs to write files to disk and needs a persistent volume to do so. - -The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. - -```yaml -… -volumes: - - name: n8n-claim0 - persistentVolumeClaim: - claimName: n8n-claim0 -… -``` - ### Environment variables n8n needs some environment variables set to pass to the application running in the containers. From 15d1baa6a98857028040edba6c989ba463b541eb Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Tue, 20 Sep 2022 13:45:39 +0200 Subject: [PATCH 09/22] Add PVC back and explanation --- docs/hosting/server-setups/google-cloud.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 63509415c..f0ab046ba 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -87,6 +87,21 @@ The `postgres-deployment.yaml` manifest then uses the values from this manifest ## Configure n8n +### Create a volume for file storage + +While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if using the xxx authentication method, which saves a file containing the xxx into file storage during startup. + +The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. + +```yaml +… +volumes: + - name: n8n-claim0 + persistentVolumeClaim: + claimName: n8n-claim0 +… +``` + ### Environment variables n8n needs some environment variables set to pass to the application running in the containers. From 793740bdc7ab48828f44031b62387c82907ad799 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Tue, 20 Sep 2022 19:03:56 +0200 Subject: [PATCH 10/22] Update details --- docs/hosting/server-setups/google-cloud.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index f0ab046ba..dfa31c331 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -88,8 +88,8 @@ The `postgres-deployment.yaml` manifest then uses the values from this manifest ## Configure n8n ### Create a volume for file storage - -While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if using the xxx authentication method, which saves a file containing the xxx into file storage during startup. + +While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if you want to persist [manual n8n encryption keys](https://docs.n8n.io/hosting/configuration/#encryption-key) between restarts, which saves a file containing the key into file storage during startup. The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. From 3b2c534a972ae588c48b96af802d51d1306cd8eb Mon Sep 17 00:00:00 2001 From: Deborah Date: Wed, 21 Sep 2022 14:53:29 +0100 Subject: [PATCH 11/22] Update docs/hosting/server-setups/google-cloud.md Co-authored-by: Chris Chinchilla --- docs/hosting/server-setups/google-cloud.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index dfa31c331..5f6f88566 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -26,7 +26,7 @@ Enable the Kubernetes Engine API by clicking the **Enable** button. ## Create a cluster -From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. +From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. You can leave the cluster configuration on defaults unless there's anything specifically you need to change, such as location. ## Set Kubectl context From f4f19ce337e4e20dd1568534f417f2463b13effa Mon Sep 17 00:00:00 2001 From: Deborah Barnard Date: Wed, 21 Sep 2022 15:06:19 +0100 Subject: [PATCH 12/22] wip --- docs/hosting/server-setups/google-cloud.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index dfa31c331..336b75572 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -4,11 +4,12 @@ This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses ## Prerequisites -- [The gcloud command line tool](https://cloud.google.com/sdk/gcloud/){:target="_blank" .external-link} +- The [gcloud command line tool](https://cloud.google.com/sdk/gcloud/){:target="_blank" .external-link} +- The [gke-gcloud-auth-plugin]{:target="_blank" .external-link} (install the gcloud CLI first) ## Hosting options -Google Cloud offers several ways suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes Engine (containers running with Kubernetes). +Google Cloud offers several options suitable for hosting n8n, including Cloud Run (optimized for running containers), Compute Engine (VMs), and Kubernetes Engine (containers running with Kubernetes). This guide uses the Google Kubernetes Engine (GKE) as the hosting option. Using Kubernetes requires some additional complexity and configuration, but is the best method for scaling n8n as demand changes. @@ -16,23 +17,23 @@ Most of the steps in this guide use the Google Cloud UI, but you can also use th ## Create project -GCP encourages you to create projects to logically organize resources and configuration. Create a new project for your n8n deployment by clicking the project dropdown menu and then the _NEW PROJECT_ button. Then select the newly created project and as you follow other steps in this guide, make sure you have the correct project selected. +GCP encourages you to create projects to logically organize resources and configuration. Create a new project for your n8n deployment from your Google Cloud Console: select the project dropdown menu and then the **NEW PROJECT** button. Then select the newly created project. As you follow the other steps in this guide, make sure you have the correct project selected. ## Enable the Kubernetes Engine API -GKE isn't enabled by default, search for "Kubernetes" in the top search bar and select "Kubernetes Engine" from the results. +GKE isn't enabled by default. Search for "Kubernetes" in the top search bar and select "Kubernetes Engine" from the results. -Enable the Kubernetes Engine API by clicking the **Enable** button. +Select **ENABLE** to enable the Kubernetes Engine API for this project. ## Create a cluster -From the GKE service page, click the **Clusters** menu item and then the **CREATE** button. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. +From the Kubernetes Engine service page, select **Clusters**, then select **CREATE**. Make sure you select the "Standard" cluster option, n8n doesn't work with an "Autopilot" cluster. ## Set Kubectl context -The remainder of the steps in this guide require you to set the GCP instance as the Kubectl context. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. +The rest of the steps in this guide require you to set the GCP instance as the Kubectl context. You can find the connection details for a cluster instance by opening its details page and selecting **CONNECT**. The displayed code snippet shows a connection string for the gcloud CLI tool. Paste and run the code snippet in the gcloud CLI to change your local Kubernetes settings to use the new gcloud cluster. -## Setup DNS +## Set up DNS n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. From 723848fde050ce439c1f905bf71db3808e4cf6ec Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Wed, 21 Sep 2022 16:58:35 +0200 Subject: [PATCH 13/22] Update docs/hosting/server-setups/google-cloud.md --- docs/hosting/server-setups/google-cloud.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 5f6f88566..9ba3875f9 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -38,7 +38,6 @@ n8n typically operates on a subdomain. Create a DNS record with your provider fo If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". - From d5dcd803fc3552f388c78a90e992597c504c681b Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Thu, 22 Sep 2022 09:09:02 +0200 Subject: [PATCH 14/22] Update IP details --- docs/hosting/server-setups/google-cloud.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index dfa31c331..b8fbb3be7 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -34,14 +34,11 @@ The remainder of the steps in this guide require you to set the GCP instance as ## Setup DNS -n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to a static IP address of the instance. +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the n8n service. Find the IP address of the n8n service from the **Services & Ingress** menu item under the **Endpoints** column. -If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". +!!! note "GKE and IP addresses" - - + [Read this GKE tutorial](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#configuring_your_domain_name_records) for more details on how reserved IP addresses work with GKE and Kubernetes resources. ## Clone configuration repository From 1f03fd069d40f13494bb7cdd5e6e2b4ed2c2bd2a Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Thu, 22 Sep 2022 09:12:26 +0200 Subject: [PATCH 15/22] Formatting --- docs/hosting/server-setups/google-cloud.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 9c9359c71..08f9522e8 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -38,7 +38,7 @@ n8n typically operates on a subdomain. Create a DNS record with your provider fo !!! note "GKE and IP addresses" - [Read this GKE tutorial](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#configuring_your_domain_name_records) for more details on how reserved IP addresses work with GKE and Kubernetes resources. + [Read this GKE tutorial](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#configuring_your_domain_name_records){:target="_blank" .external-link} for more details on how reserved IP addresses work with GKE and Kubernetes resources. ## Clone configuration repository @@ -86,7 +86,7 @@ The `postgres-deployment.yaml` manifest then uses the values from this manifest ### Create a volume for file storage -While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if you want to persist [manual n8n encryption keys](https://docs.n8n.io/hosting/configuration/#encryption-key) between restarts, which saves a file containing the key into file storage during startup. +While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if you want to persist [manual n8n encryption keys](https://docs.n8n.io/hosting/configuration/#encryption-key){:target="_blank" .external-link} between restarts, which saves a file containing the key into file storage during startup. The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. From 23d3ad1a376998270a0b5b24d7bcd7e19e417d11 Mon Sep 17 00:00:00 2001 From: Deborah Barnard Date: Mon, 26 Sep 2022 10:34:50 +0100 Subject: [PATCH 16/22] delete coment --- docs/hosting/server-setups/google-cloud.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 336b75572..422c823fd 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -39,10 +39,6 @@ n8n typically operates on a subdomain. Create a DNS record with your provider fo If the instance doesn't already have a static IP address, you can assign one to it by editing the instance, and changing the network interface from "Ephemeral" to "Static". - - ## Clone configuration repository From 4a5bde94f575660ded7a976f0efd95a29f33b670 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 26 Sep 2022 14:04:56 +0200 Subject: [PATCH 17/22] Clarify --- docs/hosting/server-setups/google-cloud.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 08f9522e8..f2c551156 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -34,7 +34,7 @@ The remainder of the steps in this guide require you to set the GCP instance as ## Setup DNS -n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the n8n service. Find the IP address of the n8n service from the **Services & Ingress** menu item under the **Endpoints** column. +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the n8n service. Find the IP address of the n8n service from the **Services & Ingress** menu item of the cluster you want to use under the **Endpoints** column. !!! note "GKE and IP addresses" From 9564421a77c82c889b56603e420ea4e1b2bc72b1 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 26 Sep 2022 14:22:18 +0200 Subject: [PATCH 18/22] Reorder --- docs/hosting/server-setups/google-cloud.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index f2c551156..04be4b351 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -32,14 +32,6 @@ From the GKE service page, click the **Clusters** menu item and then the **CREAT The remainder of the steps in this guide require you to set the GCP instance as the Kubectl context. You can find the connection details for a cluster instance by opening its details page and then the **CONNECT** button. The resulting code snippet shows a connection string for the gcloud CLI tool. Paste and run that code snippet into a terminal to change your local Kubernetes settings to use the new gcloud cluster. -## Setup DNS - -n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the n8n service. Find the IP address of the n8n service from the **Services & Ingress** menu item of the cluster you want to use under the **Endpoints** column. - -!!! note "GKE and IP addresses" - - [Read this GKE tutorial](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#configuring_your_domain_name_records){:target="_blank" .external-link} for more details on how reserved IP addresses work with GKE and Kubernetes resources. - ## Clone configuration repository Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp){:target=_blank .external-link} locally. The following steps will tell you which file configures what and what you need to change. @@ -136,6 +128,16 @@ kubectl apply -f . kubectl apply -f namespace.yaml ``` +## Setup DNS + +n8n typically operates on a subdomain. Create a DNS record with your provider for the subdomain and point it to the IP address of the n8n service. Find the IP address of the n8n service from the **Services & Ingress** menu item of the cluster you want to use under the **Endpoints** column. + +!!! note "GKE and IP addresses" + + [Read this GKE tutorial](https://cloud.google.com/kubernetes-engine/docs/tutorials/configuring-domain-name-static-ip#configuring_your_domain_name_records){:target="_blank" .external-link} for more details on how reserved IP addresses work with GKE and Kubernetes resources. + +## Delete resources + Remove the resources created by the manifests with the following command: ```shell From cb1599ed24abb02188671c2fe237155f3fc5e83d Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Wed, 28 Sep 2022 13:38:55 +0200 Subject: [PATCH 19/22] Update git info --- docs/hosting/server-setups/google-cloud.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 04be4b351..12aaea677 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -39,7 +39,7 @@ Kubernetes and n8n require a series of configuration files. You can clone these Clone the repository with the following command: ```shell -git clone https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp +git clone https://github.com/n8n-io/n8n-kubernetes-hosting.git -b gcp ``` And change directory to the root of the repository you cloned: From 998b6bd78220c953993cb94ec8b389e98ac21a3d Mon Sep 17 00:00:00 2001 From: Deborah Barnard Date: Thu, 29 Sep 2022 12:59:39 +0100 Subject: [PATCH 20/22] wip --- docs/hosting/server-setups/google-cloud.md | 23 ++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 2dae97f03..8247ec9e7 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -35,7 +35,7 @@ The rest of the steps in this guide require you to set the GCP instance as the K ## Clone configuration repository -Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp){:target=_blank .external-link} locally. The following steps will tell you which file configures what and what you need to change. +Kubernetes and n8n require a series of configuration files. You can clone these from [this repository](https://github.com/n8n-io/n8n-kubernetes-hosting/tree/gcp){:target=_blank .external-link} locally. The following steps explain the file configuration and how to add your information. Clone the repository with the following command: @@ -46,7 +46,7 @@ git clone https://github.com/n8n-io/n8n-kubernetes-hosting.git -b gcp And change directory to the root of the repository you cloned: ```shell -cd gcp +cd n8n-kubernetes-hosting ``` ## Configure Postgres @@ -55,7 +55,7 @@ For larger scale n8n deployments, Postgres provides a more robust database backe ### Create a volume for persistent storage -To maintain data between pod restarts, the Postgres deployment needs a persistent volume. Running Postgres on GCP requires a specific Kubernetes Storage Class, [you can read this guide for specifics](https://cloud.google.com/architecture/deploying-highly-available-postgresql-with-gke){:target="_blank" .external-link}, but the `storage.yaml` manifest creates it for you. You may want to change the regions to create the storage in under the `allowedTopologies` > `matchedLabelExpressions` > `values` key. By default, they're set to "us-central". +To maintain data between pod restarts, the Postgres deployment needs a persistent volume. Running Postgres on GCP requires a specific Kubernetes Storage Class. You can read [this guide](https://cloud.google.com/architecture/deploying-highly-available-postgresql-with-gke){:target="_blank" .external-link} for specifics, but the `storage.yaml` manifest creates it for you. You may want to change the regions to create the storage in under the `allowedTopologies` > `matchedLabelExpressions` > `values` key. By default, they're set to "us-central". ```yaml … @@ -67,11 +67,11 @@ allowedTopologies: - us-central1-c ``` -### Environment variables +### Postgres environment variables Postgres needs some environment variables set to pass to the application running in the containers. -The example `postgres-secret.yaml` file contains placeholders you need to replace with values of your own for user details and the database to use. +The example `postgres-secret.yaml` file contains placeholders you need to replace with your own values. Postgres will use these details when creating the database.. The `postgres-deployment.yaml` manifest then uses the values from this manifest file to send to the application pods. @@ -79,7 +79,10 @@ The `postgres-deployment.yaml` manifest then uses the values from this manifest ### Create a volume for file storage -While not essential for running n8n, using persistent volumes helps maintain files uploaded while using n8n and if you want to persist [manual n8n encryption keys](https://docs.n8n.io/hosting/configuration/#encryption-key){:target="_blank" .external-link} between restarts, which saves a file containing the key into file storage during startup. +While not essential for running n8n, using persistent volumes is required for: + +* Using nodes that interact with files, such as the binary data node. +* If you want too persist [manual n8n encryption keys](https://docs.n8n.io/hosting/configuration/#encryption-key) between restarts. This saves a file containing the key into file storage during startup. The `n8n-claim0-persistentvolumeclaim.yaml` manifest creates this, and the n8n Deployment mounts that claim in the `volumes` section of the `n8n-deployment.yaml` manifest. @@ -92,7 +95,7 @@ volumes: … ``` -### Environment variables +### n8n environment variables n8n needs some environment variables set to pass to the application running in the containers. @@ -106,13 +109,13 @@ The manifests define the following: - Send the environment variables defined to each application pod - Define the container image to use -- Set resource consumption limits. This is left empty in the example manifests, but you should set them to something appropriate for your deployment. +- Set resource consumption limits with the `resources` object. This is empty in the example manifests. You must set them to something appropriate for your deployment. - The `volumes` defined earlier and `volumeMounts` to define the path in the container to mount volumes. -- Scaling and restart policies. The example manifests define only one instance of each pod, you should change this to meet your needs. +- Scaling and restart policies. The example manifests define one instance of each pod. You should change this to meet your needs. ## Services -The two service manifests (`postgres-service.yaml` and `n8n-service.yaml`) expose the services to the outside world using the Kubernetes load balancer using ports 5432 and 5678 respectively by default. +The two service manifests (`postgres-service.yaml` and `n8n-service.yaml`) expose the services to the outside world using the Kubernetes load balancer using ports 5432 and 5678 respectively. ## Send to Kubernetes cluster From ded7db5edbafcdae09f6c50bd37a6b42ef1982b2 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Sun, 2 Oct 2022 15:18:49 +0300 Subject: [PATCH 21/22] Add resources --- docs/hosting/server-setups/google-cloud.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index 12aaea677..7b2d6af65 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -91,6 +91,26 @@ volumes: … ``` +### Pod resources + +[Kubernetes lets you](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) optionally specify the minimum resources application containers need and the limits they can run to. The example YAML files cloned above contain the following in the `resources` section of the `n8n-deployment.yaml` file: + +```yaml +… +resources: + requests: + memory: "250Mi" + limits: + memory: "500Mi" +… +``` + +This defines a minimum of 250mb per container, a maximum of 500mb, and lets Kubernetes handle CPU. You can change these values to match your own needs. As a guide, here are the resources values for the n8n cloud offerings: + +- **Start**: 320mb RAM, 10 millicore CPU burstable +- **Pro**: 640mb RAM, 20 millicore CPU burstable +- **Power**: 1280mb RAM, 80 millicore CPU burstable + ### Environment variables n8n needs some environment variables set to pass to the application running in the containers. From 384a1a4bfaaf43b1a04e470c7891152c2a199208 Mon Sep 17 00:00:00 2001 From: Deborah Barnard Date: Wed, 5 Oct 2022 12:06:28 +0100 Subject: [PATCH 22/22] fix link --- docs/hosting/server-setups/google-cloud.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/hosting/server-setups/google-cloud.md b/docs/hosting/server-setups/google-cloud.md index a059e70a4..5a6efd41d 100644 --- a/docs/hosting/server-setups/google-cloud.md +++ b/docs/hosting/server-setups/google-cloud.md @@ -5,7 +5,7 @@ This hosting guide shows you how to self-host n8n on Google Cloud (GCP). It uses ## Prerequisites - The [gcloud command line tool](https://cloud.google.com/sdk/gcloud/){:target="_blank" .external-link} -- The [gke-gcloud-auth-plugin]{:target="_blank" .external-link} (install the gcloud CLI first) +- The [gke-gcloud-auth-plugin](https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke){:target="_blank" .external-link} (install the gcloud CLI first) ## Hosting options