Self-Serve Kubernetes Clusters in Wayfinder

To get your application running on Wayfinder, you need a cluster. A cluster represents a dedicated, isolated Kubernetes cluster available for your workspace. It can be hosted in AWS, Google Cloud, or Microsoft Azure. Once you are in a Wayfinder workspace, you can access a cluster, and create one if needed.

This topic contains information on accessing existing clusters and creating new clusters, followed by information on more advanced cluster settings. Users need not be concerned with clusters as they will make use of environments. An environment maps to a Kubernetes namespace and Wayfinder will create the namespace and cluster in the background when users create their environments. Administrators can create and manage clusters directly.

See also: Managing Multi-tenant Clusters

CLI quick reference

This page shows how to manage clusters and namespaces, generally using the UI. Here is a quick reference to the equivalent CLI commands:

Task	Command
List clusters	wf get cluster
Access a cluster	wf access cluster CLUSTER-NAME See also Access a cluster
Create a cluster	wf create cluster See also Create a cluster using the CLI
Create a namespace	wf create namespace
Delete a cluster	wf delete cluster CLUSTER-NAME
Delete a namespace	wf delete namespace NAMESPACE-NAME

Access a cluster

Important

You must assume a role to access a cluster for any purpose. Role assumption is time-limited to one hour by default. After an hour you must re-assume the role, as described below, to continue working with your cluster. For more information, see Assuming Roles.

If you have one or more clusters available to your workspace, you can get kubectl access to them using the wf access cluster command.

wf access cluster does two operations (which you can also run separately):

• Sets your kubectl context to access a cluster that your workspace owns—see wf kubeconfig
• Performs a role assumption to grant you temporary permissions on that cluster—see wf assume and Assume a Role

You can provide all details directly, or run the command with no parameters to be prompted for details.

To access a cluster:

1. Run the following command:

   wf access cluster CLUSTER NAMESPACE --role ROLE

   For example:

   $ wf access cluster eks-dev project-namespace --role cluster.viewer
   ◉ Waiting for role to be applied
   ✔ Access to cluster eks-dev with role cluster.viewer granted until: 30 Apr 21 19:48 BST
   ✔ Current kubectl context set to devs.eks-dev

2. Use kubectl to access your cluster. For example:

   $ kubectl get pods -n project-namespace
   No resources found in project-namespace namespace.

Create a cluster

To create a cluster, you will need to have access to a cloud allocated to your workspace by your Wayfinder administrator.

A cluster is associated with a cluster plan, chosen when creating the cluster. A cluster plan represents a set of parameters that define how the cluster should be built: which features are enabled, what size it should be, etc.

Cluster plans:

• Provide sane default settings out of the box that reflect best practices for production and nonproduction environments.
• Remove the need for domain knowledge in development teams. They can focus on deploying their applications to staging, dev, and production environments, rather than on Kubernetes cluster types.
• Provide guard rails for the environment options workspace members can select to keep infrastructure in line with the organization's requirements.
• Include policies on whether various cluster settings can be edited by members of the workspace(s) the plans are allocated to.

You can customise certain cluster parameters. Depending on the policy applied by your Wayfinder administrator, you can change certain parameters to meet your workspace's requirements.

Services included in each cluster

Clusters created in Wayfinder come with the following pre-provisioned services:

• NGINX ingress (optional)
• Flux Helm Controller
• ExternalDNS
• cert-manager
• Auto-scalers: Installed for EKS, and enabled in GKE and AKS
• AKS: AAD Pod Identity
• EKS: Kubernetes metrics-server (available by default in GKE and AKS)
• EKS: Calico networking

If you're using multi-tenant clusters

For details on multi-tenancy in Wayfinder, see Managing Multi-tenant Clusters.

Create a cluster using Wayfinder's web interface

To create a cluster using the UI:

1. Select Admin > Kubernetes > Clusters, then click on the Create cluster button.
2. Fill in the details as outlined in the table below:

Category	Field	Description
Cluster settings	Workspaces	Select in which workspaces this plan should be available in
	Cloud provider	Select a cloud provider. If you don't see the cloud you wish to use, contact your Wayfinder administrator to have one allocated to you.
	Cluster Stage	Select a stage based on whether you're using this cluster for production of other purposes
	Cluster Subscription / Account / Project	Select a value from the drop-down menu
	Cluster Plan	Select a value from the drop-down menu
	Cluster Name	Optionally, change the cluster name
	Show advanced options	If you select Show advanced options you see advanced options that you are permitted to adjust. Otherwise, default options are used.
ADVANCED OPTIONS
	Description	Select an appropriate description for the cluster.
	Labels	Provide a key and value for the label
	Region	Select a region for the cluster
	Version	Accept the default Kubernetes version (recommended) or select a different one.
	Lifetime	Select whether you want this cluster to be deleted after an amount of time. If yes, enter a time interval. The expiry time is displayed on the Environment when developers create environments during the application self-service process.
	Private cluster	You can enable this as a private cluster. For details, see How to set up private clusters for your cloud provider.
Nodes	Node pools	Add, edit, or delete node pool configurations as needed.
Upgrades	Auto-upgrade	You can enable auto-upgrade of Kubernetes on the cluster. Also see K8s Upgrades
Multi-tenancy	Enable quota limits	For multi-tenant clusters. When enabled, this lets you add resource quota templates for tenant namespaces. These templates are similar on both the cluster plan and the cluster settings, except that as a Wayfinder administrator, you can decide whether to allow workspaces to change the template settings. For details, see Set resource quotas and constraints in the Multi-Tenancy topic.
Networks	Authorized networks	Configure the networks allowed to connect to the cluster.
	Authorized master networks	Configure the networks allowed to speak to the control plane. If left blank, this defaults to all networks.
Profiles	Configure cluster profiles	Configure cluster profiles for Linux VMs and/or Windows VMs
Pod Security Standards	Enable Pod Security Standard	Choose which policies you want to allow
Estimated Costs	Estimated Costs	View estimated costs for this plan
Cloud Specific	Remaining settings	The rest of the settings are dependent on the cloud provider–configure as needed.

important

Network settings for the cluster are not changeable once the cluster is created. To set these, see Configure cluster network settings below.

1. Once you have selected a plan, and adjusted any advanced options if desired/permitted, click Create. At this point the cluster creation process begins.

2. You can create namespaces in a cluster while it is being created.

   1. Enter a name.
   2. Select a PSS profile from the dropdown.

3. Click Create to create the namespace or click Cancel to create a namespace later.

Once you're finished, the Clusters page is displayed, where you can see the progress of the cluster creation.

Create a cluster using the CLI

When building a cluster with the CLI you must specify a cloud access configuration to use that are associated with your workspace. You must also provide a cluster plan. Contact your Wayfinder administrator if you do not have any allocated cloud access configuration or cluster plans associated with your workspace.

View Cloud Access Configuration and Cluster Plans

Use wf get cloudaccessconfig -w WORKSPACE-NAME to view the cloud access configuration.

➜  ~ wf get cloudaccessconfig -w app
NAME             PROVIDER    STATUS     IDENTIFIER                              AGE
azure-nonprod    azure       Success    6a539b80-ed1c-4fcc-b5e4-296a08bdc928    11d
➜  ~

Use wf get clusterplans -w WORKSPACE-NAME to view the available cluster plans for your workspace. If you wish to understand all of the parameters included in a plan, run wf get clusterplans PLAN-NAME -o yaml

➜  ~ wf get clusterplans -w app
NAME                    SUMMARY                                                                                               CLOUD        AGE
aks-hardened            Hardened AKS cluster with a default "restricted" PSS Policy, recommended for Production workloads.    AKS          58d
aks-playground          Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days.                AKS          58d
aks-playground-copy     Copy                                                                                                  AKS          4d3h
aks-playground.copy2    Copy 2                                                                                                AKS          4d3h
aks-standard            General purpose AKS cluster.                                                                          AKS          58d
eks-gpu                 Cluster plans which contains GPU enabled nodepools                                                    EKS          8d
eks-hardened            Hardened EKS cluster with a default "restricted" PSS Policy, recommended for Production workloads.    EKS          58d
eks-playground          Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days.                EKS          58d
eks-standard            General purpose EKS cluster.                                                                          EKS          58d
gke-hardened            Hardened GKE cluster with a default "restricted" PSS Policy, recommended for Production workloads.    GKE          58d
gke-playground          Low cost cluster configuration for testing purposes, default expiry TTL set to 7 days.                GKE          58d
gke-standard            General purpose GKE cluster.                                                                          GKE          58d

Use wf get stage to list all available stages.

NAME       DESCRIPTION                                              AGE
nonprod    Used for non-production environments, eg. development    58d
prod       Used for production environments                         58d

Create a cluster

Use wf create cluster CLUSTER-NAME -w WORKSPACE-NAME -p PLAN-NAME -a CLOUD-ACCESS-CONFIGURATION-NAME -s STAGE-NAME to create a cluster.

wf create cluster mycluster -w app -p aks-playground -a azure-nonprod -s nonprod

List all clusters

Use wf get clusters -w WORKSPACE-NAME to view all clusters in your workspace. Use the -o yaml flags to view more details about a specific cluster.

➜  ~ wf get clusters -w app
NAME (UNIQUE)    PROVIDER    PLAN              STATUS     AGE     TENANCY                       OWNER WORKSPACE
aks-opdar        AKS         aks-standard      Success    10d     Dedicated / single-tenancy    app
mycluster        AKS         aks-playground    Success    7m2s    Dedicated / single-tenancy    app

Make a cluster ephemeral (temporary)

Ephemeral clusters are deleted after a set time. You can make a cluster ephemeral when creating it, or afterward.

To make a cluster ephemeral:

Set an expiry time for the cluster in one of these two ways:

• When creating the cluster
  After entering the basic information on the cluster, click Show advanced options, set Lifetime to Set, and then enter the cluster's time to live (TTL) in the fields provided.
• After creating the cluster
  Click Admin > Kubernetes > Clusters, select your cluster, select the Settings Tab, and then click Edit. Set Lifetime to Set, and then enter the cluster's time to live (TTL) in the fields provided. After this time interval the cluster is deleted.

Manage clusters

Namespaces

When developers create a new environment and choose to use existing infrastructure, then Wayfinder will automatically create a new namespace on an existing cluster. This will keep different environments isolated from each other when developers deploy their applications into that environment.

Edit cluster settings

To edit cluster settings in Wayfinder's web interface

1. Select Admin > Kubernetes > Clusters, and then click the name of the cluster you want to edit.
2. Click the Settings tab for this cluster, and then click the Edit button.
3. Make your changes, and then click Save.

Manage cluster labels

Labels are key:value pairs in Kubernetes. You can use cluster labels in user access policies to constrain the policies to include/exclude clusters with specific labels. You can also use labels to automate software package installations across multiple clusters. For more information on these use cases, see:

• Creating Access Policies
• Package Installation

To manage labels for a cluster:

In Wayfinder's web interface, select Admin > Kubernetes > Clusters, select your cluster, select Settings, and then click the Edit button. Edit Labels as needed and click Save.

note

All clusters have the internal labels which are automatically added by Wayfinder. You can use these as well as your own labels to filter clusters as needed.

Create spot/preemptible node pools

Spot or preemptible VMs provide a cost-saving way to run workloads that can tolerate interruption, because spot instances are not guaranteed to be available from the cloud provider. For example, they are appropriate for use with batch processing jobs, or fault-tolerant testing environments.

Currently, you can add spot or preemptible node pools on GCP and Azure clusters.

For more information, see:

• GCP documentation
• Azure documentation

To add a spot nodepool:

1. Select Settings > Clusters, select your cluster, select Settings, and then click the Edit button. Edit Nodepools as needed and click Save.

   • For Azure, in the Mode field, you must select User, and turn on the Spot button.
   • For GCP, turn on the Preemptible button.

Control which IPs can access your cluster

An IP whitelist may already be in place at the cluster plan level. However, if the Wayfinder admin allows it, you can create an IP whitelist to specify that only specific IPs can access your cluster and its resources. An IP whitelist may already be in place at the cluster plan level. When developers define their container components during the application creation process, they can specify a whitelist for that container component.

To create an IP whitelist:

1. Select Settings > Clusters, select your cluster, select Settings, and then click the Edit button. Edit Networks as needed and click Save.
2. The existing default setting lets all networks access the cluster.
3. For every IP permitted to access your cluster, click Add network, and then enter a name and the IP address.

caution

Be sure to remove the default setting.

Configure cluster network settings

There are two factors that determine your options for network settings on a cluster:

• You can change default network settings if the cluster plan allows it.
• Your Wayfinder administrator can configure network IP range allocations. If these have been configured, the cluster network ranges can be auto-assigned, or you can provide a custom IP range.

The network types for which you can provide IP ranges are:

Cloud	Supported network types
EKS	Nodes
AKS	Nodes, Services
GKE	Nodes, Services, Pods

important

The cluster network settings cannot be changed once the cluster is created.

Upgrade Kubernetes on a cluster

The Wayfinder GUI supports upgrades at the cluster level for control planes and node pools.

If the Wayfinder administrator allows it, you can select whether to upgrade the Kubernetes version on a cluster either automatically or manually. The option to enable auto-upgrade is available when you create or edit a cluster.

Delete a cluster

Deleting a cluster also deletes namespaces on that cluster, including tenant namespaces if it's a multi-tenant cluster.

To delete a cluster:

Wayfinder's web interface: Select Settings > Clusters, select the three vertical dots to see more options, then select Delete Cluster.

CLI: wf delete cluster CLUSTER-NAME