Cluster Plans
Cluster plans act as templates for self-serve clusters. The plans serve as guardrails that let Wayfinder administrators shape the options and configuration available for developers to self-serve their environments.
As a Wayfinder administrator, you select which workspaces each cluster plan should be available in. You can specify more than one workspace for each cluster plan. Cluster plans are visible in workspaces as Environment Plans. Developers use environment plans as a template from which to provision an environment during their application deployment process. Wayfinder will provision their environment according to the settings specified in the cluster plan.
With cluster plans, you can specify cluster settings such as region, kubernetes versions, node pools and instance types, provider features such as GKE autorepair and autoscaling, as well as options related to private clusters, quota limits, upgrades, pod security standards and estimated costs.
Benefits of cluster plans
- Provide sane default settings out of the box that reflect best practices for production and nonproduction environments.
- Remove the need for domain knowledge in development teams. They can focus on deploying their applications to staging, dev, and production environments, rather than on Kubernetes cluster types.
- Provide guard rails for the environment options workspace members can select to keep infrastructure in line with the organization's requirements.
- Include policies on whether various cluster settings can be edited by members of the workspace(s) the plans are allocated to.
Cluster security
We have followed guidelines published by the cloud providers for configuring clusters:
- Removing any loose permissions bindings
- Removing previous pod security policies and defaulting all applications to least privileged
wf.defaultpolicies - Enforcing a default denial on all namespaces created—ensuring traffic is explicitly permitted rather than a wildcard to all
How cluster plans are consumed
Once the plans are set up, workspace members can select them to create an environment. Thereafter, they can deploy their application into that environment.
Examples on how to create environments using cluster plans can be found here.
Using cluster plans with YAML
When developers (as workspace members) create an environment, they can choose to use existing infrastructure (Wayfinder creates a new namespace on an existing cluster) or choose to provision new infrastructure (Wayfinder provisions a new cluster and a namespace in accordance to the cluster plan).
When provisioning an environment using new infrastructure, either using the CLI or Wayfinder's web interface, the cluster plan is used as a boilerplate of defaults for the cluster's specifications. This cluster definition is submitted to the API and verified against the cluster plan's policy, ensuring the cluster definition does not violate any restrictions in the cluster plan.
The developer is in full control of the application's configuration and selecting an environment with a cluster configuration that suits their needs. Clusters are represented
to developers as environments, so the underlying application's environment YAML definition must have a reference spec.plan (cluster plan). Developers can only specify
cluster plans that are available to them in their workspace. By default, workspace members can't create any new cluster plans or edit existing ones. Upon creating or updating
the application environment's YAML definition, the cluster definition is verified against the cluster plan's policy.
Below is an example of the developer's application environment YAML manifest: Wayfinder provisions the cluster according to the specification outlined in the aks-playground cluster plan. For more YAML examples, please see the Application Environment section of the documentation.
➜ ~ wf create appenv --app myapp --env dev3 --cluster devhost --plan aks-playground --stage nonprod --region uksouth
spec:
application: myapp
cloud: azure
clusterRef:
group: compute.appvia.io
kind: Cluster
name: devhost
namespace: ws-dmo
version: v2beta1
key: dev3
name: dev3
namespace: myapp-dev3
plan: aks-playground <-- This is the cluster plan
region: uksouth
stage: nonprod
Wayfinder's administrators have permissions to create new cluster plans and to edit existing ones. The exception to the rule is that Wayfinder's default cluster plans are read-only, but you can create a copy and update the settings as per your requirements. Please see the sections below for examples.
View cluster plans
By default, the cluster plans provided by Wayfinder for each cloud provider are available in all workspaces. Wayfinder administrators can choose if new cluster plans should be available in all workspaces or only some workspaces.
To view cluster plans in the CLI:
Run
wf get clusterplan.Cluster plans for all cloud providers are displayed:
➜ ~ wf get clusterplan
NAME SUMMARY CLOUD AGE
aks-hardened Hardened AKS cluster with a default "restricted" PSS Policy, recommended for Production workloads. AKS 51d
aks-playground Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days. AKS 51d
aks-standard General purpose AKS cluster. AKS 51d
eks-gpu Cluster plans which contains GPU enabled nodepools EKS 30h
eks-hardened Hardened EKS cluster with a default "restricted" PSS Policy, recommended for Production workloads. EKS 51d
eks-playground Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days. EKS 51d
eks-standard General purpose EKS cluster. EKS 51d
gke-hardened Hardened GKE cluster with a default "restricted" PSS Policy, recommended for Production workloads. GKE 51d
gke-playground Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days. GKE 51d
gke-standard General purpose GKE cluster. GKE 51d
To view cluster plans using Wayfinder's web interface:
Select Settings, then navigate to Developer Self-Service > Cluster plans.
Click the cloud provider that you want to see plans for, for example, Google Cloud Platform.
The plans are displayed.
Add a new cluster plan
Wayfinder's default cluster plans for each cloud provider cannot be edited. But you can add your own plans. A convenient way to add a cluster plan is to copy one, and then customise it. You can also start from scratch. Once you've added a new plan for a cloud provider and specified which workspaces the plan should be available in, workspace members in that workspace and cloud can select that plan when creating an environment.
In the cluster plan configuration, you can enforce which cluster settings can be changed by workspace owners, for example, if clusters should auto-upgrade or not.
The plan parameters, and whether they are permitted to be changed, applies to any workspace you allocate this plan to. If you want to set different permissions for specific workspaces, you must create a new plan.
If you change a plan to modify the cluster configuration, this does not modify the clusters that were built from it, but applies to new clusters using that plan. However, if you update policies in the plan, the new policies are enforced for all clusters using the plan, meaning any changes to the configuration of those clusters must match the updated policies on the plan.
Services included in each cluster
Clusters created in Wayfinder come with the following pre-provisioned services:
- NGINX ingress (optional)
- Flux Helm Controller
- ExternalDNS
- cert-manager
- Auto-scalers: Installed for EKS, and enabled in GKE and AKS
- AKS: AAD Pod Identity
- EKS: Kubernetes metrics-server (available by default in GKE and AKS)
- EKS: Calico networking
Add a cluster plan in Wayfinder's web interface
To add a cluster plan:
- Click on Settings, then navigate to Developer Self-Service > Cluster plans.
- Click the tab for the cloud provider that you want to add a plan for, and then click the Create cluster plan button.
- Set or change options as needed.
- If you want to copy an existing plan, then select the plan and then click on the Copy button.
- Enter the details. For most parameters, you can set whether to allow changes. When you allow changes, then workspace owners and members will be able to edit those parameters.
- Click Save.
| Category | Field | Description |
|---|---|---|
| Cluster plan information | Plan name & description | Select an appropriate name and description so that developers can easily choose the right plan for them when creating a cluster, for example, eks-budget-plan. |
| Workspaces | Select in which workspaces this plan should be available in | |
| Cluster settings | Labels | Provide a key and value for the label |
| Region | Select a region for the cluster | |
| Version | Accept the default Kubernetes version (recommended) or select a different one. | |
| Cluster Lifetime | Select whether you want this cluster to be deleted after an amount of time. If yes, enter a time interval. The expiry time is displayed on the Environment Plans when developers create environments during the application self-service process. | |
| Private cluster | You can enable this as a private cluster. For details, see How to set up private clusters for your cloud provider. | |
| Node pools | Add, edit, or delete node pool configurations as needed. | |
| Upgrades | Auto-upgrade | You can enable auto-upgrade of Kubernetes on the cluster. Also see K8s Upgrades |
| Multi-tenancy | Enable quota limits | For multi-tenant clusters. When enabled, this lets you add resource quota templates for tenant namespaces. These templates are similar on both the cluster plan and the cluster settings, except that as a Wayfinder administrator, you can decide whether to allow workspaces to change the template settings. For details, see Set resource quotas and constraints in the Multi-Tenancy topic. |
| Networks | Authorized networks | Configure the networks allowed to connect to the cluster. |
| Authorized master networks | Configure the networks allowed to speak to the control plane. If left blank, this defaults to all networks. | |
| Profiles | Configure cluster profiles | Configure cluster profiles for Linux VMs and/or Windows VMs |
| Pod Security Standards | Enable Pod Security Standard | Choose which policies you want to allow |
| Estimated Costs | Estimated Costs | View estimated costs for this plan |
| Cloud Specific | Remaining settings | The rest of the settings are dependent on the cloud provider–configure as needed. |
Add a cluster using the CLI
You can also create your own plan by writing a .yaml file specifying the plan parameters, and then applying it to Wayfinder using the CLI.
We recommend you start with an out-of-the-box plan and edit it to suit your needs.
To create a new cluster plan from an existing one:
Get the yaml from an existing plan:
wf get clusterplan gke-playground -o yaml > MYPLAN.yamlThis copies the the
gke-playgroundcluster plan and places it in a fileMYPLAN.yaml.Edit the plan file as needed, being sure to change the metadata name to a new name to indicate that you wish to make a new plan. Thereafter, apply the file:
wf apply -f MYPLAN.yaml
You can prevent workspaces from editing a parameter when creating a cluster by setting editable to false for that parameter. In below example
the cluster's region, version and upgrade method is locked (set to false) from being edited. The cluster's lifetime is editable (set to true).
policies:
- editable: false
path: spec.region
- editable: false
path: spec.version
- editable: true
path: spec.expires
- editable: false
path: spec.enableAutoUpgrade
Below is an example of the gke-playground cluster plan.
➜ ~ wf get clusterplan gke-playground -o yaml > MYPLAN.yaml
apiVersion: compute.appvia.io/v2beta1
kind: ClusterPlan
metadata:
annotations:
appvia.io/readonly: "true"
name: gke-playground
spec:
allocation:
type: all
workspaces:
- '*'
defaultNetworkFabricPlan: gke-standard
summary: Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days.
template:
channel: default
enableAutoUpgrade: false
enablePrivateCluster: false
expires: 168h0m0s
networking:
authorizedMasterNetworks:
- cidr: 0.0.0.0/0
name: default
authorizedNetworks:
- cidr: 0.0.0.0/0
name: default
nodePools:
- autoscaling:
enabled: true
maxSize: 10
minSize: 1
diskSize: 30
image: COS_CONTAINERD
logicalName: compute
machine: n2-standard-2
maxPodsPerNode: 110
size: 1
spot: {}
plan: gke-playground
provider: GKE
providerDetails:
gke:
enableHTTPLoadBalancer: false
enableHorizontalPodAutoscaler: true
enableShieldedNodes: false
enableStackDriverLogging: false
enableStackDriverMetrics: false
masterIPV4Cidr: 172.16.0.16/28
type: GKE
quotaLimits:
default: small
templates:
- constraints:
hardQuota:
requests.cpu: "1"
requests.memory: 3.5Gi
limitRanges:
- max:
cpu: 256m
ephemeral-storage: 1.875Gi
memory: 896Mi
type: Pod
name: small
resourceDefaults:
- default:
cpu: 256m
ephemeral-storage: 10Mi
memory: 896Mi
defaultRequest:
cpu: 50m
memory: 200Mi
type: Container
- constraints:
hardQuota:
requests.cpu: "2"
requests.memory: 7Gi
limitRanges:
- max:
cpu: 512m
ephemeral-storage: 3.75Gi
memory: 1792Mi
type: Pod
name: medium
resourceDefaults:
- default:
cpu: 512m
ephemeral-storage: 500Mi
memory: 1792Mi
defaultRequest:
cpu: 50m
memory: 200Mi
type: Container
- constraints:
hardQuota:
requests.cpu: 14Gi
requests.memory: 7Gi
limitRanges:
- max:
cpu: "1"
ephemeral-storage: 7.5Gi
memory: 3584Mi
type: Pod
name: large
resourceDefaults:
- default:
cpu: "1"
ephemeral-storage: 1Gi
memory: 1792Mi
defaultRequest:
cpu: 50m
memory: 200Mi
type: Container
region: europe-west2
security:
podSecurityStandard:
allowed:
- restricted
- baseline
- privileged
defaultProfile: baseline
enabled: true
stage: ""
version: latest
Associate a plan to a workspace
Cluster plans can be allocated to one or more workspaces using the spec.allocation field in the resource definition.
For the full plan specification, see PlanSpec.
The YAML example below specifies that the cluster plan is available in all workspaces.
spec:
allocation:
type: all
workspaces:
- '*'
The YAML example below specifies that the cluster plan is only available in the 'sandbox' workspace.
spec:
allocation:
type: workspaces
workspaces:
- sandbox
Define the management peering rule
This is only required for private clusters. Skip this if your plan does not enable a private cluster.
Wayfinder requires direct connectivity to the public kubernetes API. This is the cloud managed Kubernetes API, not the authentication module.
Setting a cluster to private (enablePrivateCluster: true in the above cluster plan) removes all public access to the API endpoint, so you must create a peering back to the
management network.
To define the management peering rule:
Create a resource file and apply it using the CLI:
wf apply -f FILENAMEHere's a sample peering rule definition:
---
apiVersion: networking.appvia.io/v1alpha1
kind: PeeringRule
metadata:
name: management
spec:
filters:
allocation:
type: all
selectors:
matchExpressions:
- key: networking.appvia.io/peering
operator: In
values: ["true"]
- key: appvia.io/provider
operator: In
values: ["aws"]
connection:
type: peering
peering:
enableAutoApproval: true