Cluster Plans

Cluster plans are like templates for self-serve clusters. The plans act as guardrails that let platform operators shape the options and configuration available for developers to self-serve their environment.

With cluster plans, you can specify settings like kubernetes versions, node pools and instance types, provider features such as GKE autorepair and autoscaling, as well as options related to the default access roles that workspaces inherit.

Benefits of cluster plans

• Provide sane default settings out of the box that reflect best practices for production and nonproduction environments.
• Remove the need for domain knowledge in development teams. They can focus on deploying their applications to staging, dev, and production environments, rather than on Kubernetes cluster types.
• Provide guard rails for the environment options workspace members can select to keep infrastructure in line with the organization's requirements.
• Include policies on whether various cluster settings can be edited by members of the workspace(s) the plans are allocated to.

Cluster security

We have followed guidelines published by the cloud providers for configuring clusters:

• Removing any loose permissions bindings
• Removing previous pod security policies and defaulting all applications to least privileged wf.default policies
• Enforcing a default denial on all namespaces created—ensuring traffic is explicitly permitted rather than a wildcard to all

How cluster plans are consumed

Once the plans are set up, workspace members can select them when building their environment. For example, to use the gke-development plan when creating a cluster, using the CLI:

wf create cluster CLUSTER-NAME --plan gke-development

In the UI, the user selects the plan as part of a new cluster setup.

Adding cluster plans with YAML

When provisioning a cluster using the CLI or UI, the cluster plan is used as a boilerplate of defaults with any user-defined options merged into the cluster's YAML definition. This cluster definition is submitted to the API and verified against the cluster plan's policy, ensuring the cluster definition does not violate any restrictions in the cluster plan.

The user is in full control of the cluster configuration, but the cluster's YAML definition must have a reference spec.plan (cluster plan). On creating or updating the cluster's YAML definition, the definition is verified against the cluster plan's policy.

View cluster plans

By default, all workspaces can access all plans, however if you have configured Wayfinder cloud account automation you can restrict this by only including certain plans in the naming rules.

To view cluster plans in the UI:

1. Navigate to Wayfinder settings > Cluster configuration > Cluster plans.

2. Click the cloud provider that you want to see plans for, for example, Microsoft Azure.

   The plans are displayed.

To view cluster plans in the CLI:

1. Run wf get plan.

   Cluster plans for all cloud providers are displayed:

   $ wf get plan
   RESOURCE                 SUMMARY                                                                DESCRIPTION                KIND    AGE
   aks-development          Provides a development AKS cluster                                     AKS Development Cluster    AKS     27d
   aks-production           Provides a production AKS cluster                                      AKS Production Cluster     AKS     27d
   eks-development          Provides a development cluster within EKS                              EKS Development Cluster    EKS     125d
   eks-production           Provides a production cluster within EKS                               EKS Production Cluster     EKS     125d
   gke-budget-cluster       Provides a cheap cluster using pre-emptible nodes, good for testing    GKE Budget Cluster         GKE     30d
   gke-development          Provides a development cluster within GKE                              GKE Development Cluster    GKE     133d
   gke-production           Provides a production cluster within GKE                               GKE Production Cluster     GKE     133d

Add a new cluster plan

The default cluster plans provided by Wayfinder for each cloud provider cannot be edited. But you can add your own plans. A convenient way to add a cluster plan is to copy one, and then customize it. You can also start from scratch. Once you've added a new plan for a cloud provider, workspace members can select that plan when creating a cluster in that cloud in Wayfinder.

In the cluster plan configuration, you can enforce which cluster settings can be changed when workspace members create the cluster using this plan. For example, you can enforce whether workspace members can enable cluster auto-upgrade, or if they can add an IP whitelist.

note

The plan parameters, and whether they are permitted to be changed, applies to any workspace you allocate this plan to. If you want to set different permissions for specific workspaces, you must create a new plan.

note

If you change a plan to modify the cluster configuration, this does not modify the clusters that were built from it, but applies to new clusters using that plan. However, if you update policies in the plan, the new policies are enforced immediately for all clusters using the plan, without changing existing cluster configuration.

Services included in each cluster

Clusters created in Wayfinder come with the following pre-provisioned services:

• NGINX ingress (optional)
• Flux Helm Controller
• ExternalDNS
• cert-manager
• Auto-scalers: Installed for EKS, and enabled in GKE and AKS
• AKS: AAD Pod Identity
• EKS: Kubernetes metrics-server (available by default in GKE and AKS)
• EKS: Calico networking

Add a cluster plan in the UI

To add a cluster plan:

1. Navigate to Wayfinder settings > Cluster configuration > Cluster plans.

2. Click the tab for the cloud provider that you want to add a plan for, and then click Add new cluster plan.

3. Either select an existing plan to copy as a starting point, or click Start from scratch.

4. Set or change options as needed (see table below).

   For most parameters, you can set whether to allow workspace members to change it in their cluster settings. Where applicable, you can also constrain changes by clicking Configure allowed values, for example:

   

   Field	Description
   Plan name/description	Select an appropriate name and description so that developers can easily choose the right plan for them when creating a cluster, for example, eks-budget-plan.
   Region	Select a region for the cluster
   Version	Accept the default Kubernetes version (recommended) or select a different one.
   Auto-upgrade	You can enable auto-upgrade of Kubernetes on the cluster.
   Maintenance window	Enter a time of day for maintenance operations (includes auto-upgrade).
   Expiry (TTL)	Select whether you want this cluster to be deleted after an amount of time. If yes, enter a time interval. The expiry time is displayed on the Clusters page.
   Private cluster	You can enable this as a private cluster. For details, see How to set up private clusters for your cloud provider.
   Authorized master networks	Configure the networks allowed to speak to the control plane. If left blank, this defaults to all networks.
   Authorized networks	Configure the networks allowed to connect to the cluster.
   Node network Services network Pod network	Enter network ranges for these networks. Supported network types depend on the cloud provider, and whether or not you have allocated IP address ranges for this cloud provider. If you have allocated IP address ranges, you can auto-assign the network range(s), or use a custom range. For more information, see Add a network allocation.
   Addons	Enable additional features.
   Node pools	Add, edit, or delete node pool configurations as needed.
   Enable quota limits	For multi-tenant clusters. When enabled, this lets you add resource quota templates for tenant namespaces. These templates are similar on both the cluster plan and the cluster settings, except that as a Wayfinder administrator, you can decide whether to allow workspaces to change the template settings. For details, see Set resource quotas and constraints in the Multi-Tenancy topic.
   Remaining settings	The rest of the settings are dependent on the cloud provider–configure as needed.

5. Click Save.

Manually add a new cluster plan

You can also create your own plan by writing a .yaml file specifying the plan parameters, and then applying it to Wayfinder using the CLI. We recommend you start with an out-of-the-box plan and edit it to suit your needs. You will need some knowledge of the Open Policy Agent (OPA).

To create a new cluster plan from an existing one:

1. Get the yaml from an existing plan:

   wf get plan gke -o yaml > MYPLAN.yaml

   This copies the the gke cluster plan and places it in a file MYPLAN.yaml.

2. Edit the plan file as needed, and then apply the file:

   wf apply -f MYPLAN.yaml

   Below is a sample plan.

   • You can prevent workspaces from editing a parameter when creating a cluster by setting editable to false for that parameter.

   • You can control which capabilities will be enabled or disabled by default in the capabilities section of the template. Workspaces can optionally enable/disable capabilities in their cluster. For more information, see Make global package a capability.

   apiVersion: compute.appvia.io/v1alpha1
   kind: Plan
   metadata:
     name: eks-default
     namespace: ws-admin
   spec:
     allocation:
       type: all
     policies:
     - editable: false
       path: spec.networking.authorizedNetworks
     - editable: false
       path: spec.networking.authorizedMasterNetworks
     - editable: false
       path: spec.enablePrivateCluster
     - editable: false
       path: spec.networking.clusterIPV4Cidr
     summary: Default EKS Template
     template:
       capabilities:
         psa-operator:
           description: "Pod Security Admission Operator"
           enabled: true
         ingress-nginx:
           description: "Ingress Nginx"
           enabled: true
         terranetes-controller:
           description: "Terranetes Controller"
           enabled: true
         cert-manager:
           description: "Certificate Manager operator"
           enabled: true
         external-dns-aws:
           description: "External DNS (AWS)"
           enabled: true
       enablePrivateCluster: true
       networking:
         authorizedMasterNetworks:
         - cidr: 0.0.0.0/0
           name: default
         authorizedNetworks:
         - cidr: 10.70.0.0/24
           name: ci
         - cidr: 10.80.0.0/24
           name: vpn
         clusterIPV4Cidr: auto
       nodePools:
       - autoscaling:
           enabled: true
           maxSize: 10
           minSize: 1
         diskSize: 30
         image: AL2_x86_64
         machine: t3.medium
         name: compute
         size: 2
         spot:
           enabled: true
       provider: EKS
       region: eu-west-2
       version: latest

Associate a plan to a workspace

Cluster plans can be allocated to one or more workspaces using the spec.allocation field in the resource definition. For the full plan specification, see PlanSpec.

Define the management peering rule

note

This is only required for private clusters. Skip this if your plan does not enable a private cluster.

Wayfinder requires direct connectivity to the public kubernetes API. This is the cloud managed Kubernetes API, not the authentication module. Setting a cluster to private (enablePrivateCluster: true in the above cluster plan) removes all public access to the API endpoint, so you must create a peering back to the management network.

To define the management peering rule:

1. Create a resource file and apply it using the CLI:

   wf apply -f FILENAME

   Here's a sample peering rule definition:

   ---
   apiVersion: networking.appvia.io/v1alpha1
   kind: PeeringRule
   metadata:
     name: management
   spec:
     filters:
       allocation:
         type: all
       selectors:
         matchExpressions:
           - key: networking.appvia.io/peering
             operator: In
             values: ["true"]
           - key: appvia.io/provider
             operator: In
             values: ["aws"]
     connection:
       type: peering
       peering:
         enableAutoApproval: true