Cluster Plans
Cluster plans are like templates for self-serve clusters. The plans act as guardrails that let platform operators shape the options and configuration available for developers to self-serve their environment.
With cluster plans, you can specify settings like kubernetes versions, node pools and instance types, provider features such as GKE autorepair and autoscaling, as well as options related to the default access roles that workspaces inherit.
Benefits of cluster plans​
- Provide sane default settings out of the box that reflect best practices for production and nonproduction environments.
- Remove the need for domain knowledge in development teams. They can focus on deploying their applications to staging, dev, and production environments, rather than on Kubernetes cluster types.
- Provide guard rails for the environment options workspace members can select to keep infrastructure in line with the organization's requirements.
- Include policies on whether various cluster settings can be edited by members of the workspace(s) the plans are allocated to.
Cluster security​
We have followed guidelines published by the cloud providers for configuring clusters:
- Removing any loose permissions bindings
- Removing previous pod security policies and defaulting all applications to least privileged
wf.default
policies - Enforcing a default denial on all namespaces created—ensuring traffic is explicitly permitted rather than a wildcard to all
How cluster plans are consumed​
Once the plans are set up, workspace members can select them when building their environment. For example, to use the gke-development
plan when creating a cluster, using the CLI:
wf create cluster CLUSTER-NAME --plan gke-development
In the UI, the user selects the plan as part of a new cluster setup.
Adding cluster plans with YAML​
When provisioning a cluster using the CLI or UI, the cluster plan is used as a boilerplate of defaults with any user-defined options merged into the cluster's YAML definition. This cluster definition is submitted to the API and verified against the cluster plan's policy, ensuring the cluster definition does not violate any restrictions in the cluster plan.
The user is in full control of the cluster configuration, but the cluster's YAML definition must have a reference spec.plan
(cluster plan). On creating or updating the cluster's YAML definition, the definition is verified against the cluster plan's policy.
View cluster plans​
By default, all workspaces can access all plans, however if you have configured Wayfinder cloud account automation you can restrict this by only including certain plans in the naming rules.
To view cluster plans in the UI:
Navigate to Wayfinder settings > Cluster configuration > Cluster plans.
Click the cloud provider that you want to see plans for, for example, Microsoft Azure.
The plans are displayed.
To view cluster plans in the CLI:
Run
wf get plan
.Cluster plans for all cloud providers are displayed:
$ wf get plan
RESOURCE SUMMARY DESCRIPTION KIND AGE
aks-development Provides a development AKS cluster AKS Development Cluster AKS 27d
aks-production Provides a production AKS cluster AKS Production Cluster AKS 27d
eks-development Provides a development cluster within EKS EKS Development Cluster EKS 125d
eks-production Provides a production cluster within EKS EKS Production Cluster EKS 125d
gke-budget-cluster Provides a cheap cluster using pre-emptible nodes, good for testing GKE Budget Cluster GKE 30d
gke-development Provides a development cluster within GKE GKE Development Cluster GKE 133d
gke-production Provides a production cluster within GKE GKE Production Cluster GKE 133d
Add a new cluster plan​
The default cluster plans provided by Wayfinder for each cloud provider cannot be edited. But you can add your own plans. A convenient way to add a cluster plan is to copy one, and then customize it. You can also start from scratch. Once you've added a new plan for a cloud provider, workspace members can select that plan when creating a cluster in that cloud in Wayfinder.
In the cluster plan configuration, you can enforce which cluster settings can be changed when workspace members create the cluster using this plan. For example, you can enforce whether workspace members can enable cluster auto-upgrade, or if they can add an IP whitelist.
note
The plan parameters, and whether they are permitted to be changed, applies to any workspace you allocate this plan to. If you want to set different permissions for specific workspaces, you must create a new plan.
note
If you change a plan to modify the cluster configuration, this does not modify the clusters that were built from it, but applies to new clusters using that plan. However, if you update policies in the plan, the new policies are enforced immediately for all clusters using the plan, without changing existing cluster configuration.
Services included in each cluster​
Clusters created in Wayfinder come with the following pre-provisioned services:
- NGINX ingress (optional)
- Flux Helm Controller
- ExternalDNS
- cert-manager
- Auto-scalers: Installed for EKS, and enabled in GKE and AKS
- AKS: AAD Pod Identity
- EKS: Kubernetes metrics-server (available by default in GKE and AKS)
- EKS: Calico networking
Add a cluster plan in the UI​
To add a cluster plan:
Navigate to Wayfinder settings > Cluster configuration > Cluster plans.
Click the tab for the cloud provider that you want to add a plan for, and then click Add new cluster plan.
Either select an existing plan to copy as a starting point, or click Start from scratch.
Set or change options as needed (see table below).
For most parameters, you can set whether to allow workspace members to change it in their cluster settings. Where applicable, you can also constrain changes by clicking Configure allowed values, for example:
Field Description Plan name/description Select an appropriate name and description so that developers can easily choose the right plan for them when creating a cluster, for example, eks-budget-plan
.Region Select a region for the cluster Version Accept the default Kubernetes version (recommended) or select a different one. Auto-upgrade You can enable auto-upgrade of Kubernetes on the cluster. Maintenance window Enter a time of day for maintenance operations (includes auto-upgrade). Expiry (TTL) Select whether you want this cluster to be deleted after an amount of time. If yes, enter a time interval. The expiry time is displayed on the Clusters page. Private cluster You can enable this as a private cluster. For details, see How to set up private clusters for your cloud provider. Authorized master networks Configure the networks allowed to speak to the control plane. If left blank, this defaults to all networks. Authorized networks Configure the networks allowed to connect to the cluster. - Node network
- Services network
- Pod network
Enter network ranges for these networks. Supported network types depend on the cloud provider, and whether or not you have allocated IP address ranges for this cloud provider. If you have allocated IP address ranges, you can auto-assign the network range(s), or use a custom range. For more information, see Add a network allocation. Addons Enable additional features. Node pools Add, edit, or delete node pool configurations as needed. Enable quota limits For multi-tenant clusters. When enabled, this lets you add resource quota templates for tenant namespaces. These templates are similar on both the cluster plan and the cluster settings, except that as a Wayfinder administrator, you can decide whether to allow workspaces to change the template settings. For details, see Set resource quotas and constraints in the Multi-Tenancy topic. Remaining settings The rest of the settings are dependent on the cloud provider–configure as needed. Click Save.
Manually add a new cluster plan​
You can also create your own plan by writing a .yaml
file specifying the plan parameters, and then applying it to Wayfinder using the CLI. We recommend you start with an out-of-the-box plan and edit it to suit your needs. You will need some knowledge of the Open Policy Agent (OPA).
To create a new cluster plan from an existing one:
Get the yaml from an existing plan:
wf get plan gke -o yaml > MYPLAN.yaml
This copies the the
gke
cluster plan and places it in a fileMYPLAN.yaml
.Edit the plan file as needed, and then apply the file:
wf apply -f MYPLAN.yaml
Below is a sample plan.
You can prevent workspaces from editing a parameter when creating a cluster by setting
editable
tofalse
for that parameter.You can control which capabilities will be enabled or disabled by default in the
capabilities
section of the template. Workspaces can optionally enable/disable capabilities in their cluster. For more information, see Make global package a capability.
apiVersion: compute.appvia.io/v1alpha1
kind: Plan
metadata:
name: eks-default
namespace: ws-admin
spec:
allocation:
type: all
policies:
- editable: false
path: spec.networking.authorizedNetworks
- editable: false
path: spec.networking.authorizedMasterNetworks
- editable: false
path: spec.enablePrivateCluster
- editable: false
path: spec.networking.clusterIPV4Cidr
summary: Default EKS Template
template:
capabilities:
psa-operator:
description: "Pod Security Admission Operator"
enabled: true
ingress-nginx:
description: "Ingress Nginx"
enabled: true
terranetes-controller:
description: "Terranetes Controller"
enabled: true
cert-manager:
description: "Certificate Manager operator"
enabled: true
external-dns-aws:
description: "External DNS (AWS)"
enabled: true
enablePrivateCluster: true
networking:
authorizedMasterNetworks:
- cidr: 0.0.0.0/0
name: default
authorizedNetworks:
- cidr: 10.70.0.0/24
name: ci
- cidr: 10.80.0.0/24
name: vpn
clusterIPV4Cidr: auto
nodePools:
- autoscaling:
enabled: true
maxSize: 10
minSize: 1
diskSize: 30
image: AL2_x86_64
machine: t3.medium
name: compute
size: 2
spot:
enabled: true
provider: EKS
region: eu-west-2
version: latest
Associate a plan to a workspace​
Cluster plans can be allocated to one or more workspaces using the spec.allocation
field in the resource definition. For the full plan specification, see PlanSpec.
Define the management peering rule​
note
This is only required for private clusters. Skip this if your plan does not enable a private cluster.
Wayfinder requires direct connectivity to the public kubernetes API. This is the cloud managed Kubernetes API, not the authentication module.
Setting a cluster to private (enablePrivateCluster: true
in the above cluster plan) removes all public access to the API endpoint, so you must create a peering back to the management network.
To define the management peering rule:
Create a resource file and apply it using the CLI:
wf apply -f FILENAME
Here's a sample peering rule definition:
---
apiVersion: networking.appvia.io/v1alpha1
kind: PeeringRule
metadata:
name: management
spec:
filters:
allocation:
type: all
selectors:
matchExpressions:
- key: networking.appvia.io/peering
operator: In
values: ["true"]
- key: appvia.io/provider
operator: In
values: ["aws"]
connection:
type: peering
peering:
enableAutoApproval: true