Skip to main content

How do I use Wayfinder to deploy a logging solution (like ELK)?

Prerequisites​

note

Make sure you have:

  • Wayfinder installed (version 2.7 or above)
  • Admin permissions in Wayfinder

Workflow​

Wayfinder packages simplify deploying observability solutions like ELK into your cloud environment. ELK stands for Elasticsearch, Logstash, and Kibana. Together, these components provide a powerful, integrated solution for managing large volumes of data, offering real-time insights and a comprehensive analytics suite.

In this example, we'll deploy only Elasticsearch for the backend and Kibana for the frontend. We'll use Elastic Cloud on Kubernetes (ECK), the official operator by Elastic, to automate the deployment, provisioning, management, and orchestration of Elasticsearch and Kibana on Kubernetes.


Process Diagram


The steps include:

Create:

  1. Wayfinder Repository: Give Wayfinder access to the relevant Helm Charts.
  2. Wayfinder Package: Deploy the Elasticsearch (ECK) Operator.
  3. Wayfinder Package: Deploy the Elasticsearch (ECK) Stack.

Validate:

  1. Validate your packages against a test cluster

Include:

  1. Cluster Plan: Add the two packages to a cluster plan for deployment to a cluster.

Deploy:

  1. AppEnv: Create an application environment using the cluster plan in step 5 and deploy the packages.

Instructions​

Create​

1. Create a Wayfinder Repository​

Configure Wayfinder to source the Elastic Stack Helm Charts.

  • Create a file named elastic-repository.yaml and copy the following YAML content into it:
apiVersion: package.appvia.io/v2beta2
kind: Repository
metadata:
  name: elastic-stack
spec:
  url: https://helm.elastic.co

  • Apply the YAML configuration using the wf apply command. Ensure the file is in your working directory, or specify the full path:
wf apply -f elastic-repository.yaml


2. Create a Wayfinder Package for the ECK Operator​

The ECK operator will manage the deployment, provisioning, management, and orchestration of Elasticsearch and Kibana on Kubernetes.

  • Use wf init package PACKAGE --helm-repo REPO --chart CHART --version VERSION command to create the package YAML for the ECK Operator:
`wf init package eck-operator --helm-repo https://helm.elastic.co --chart eck-operator --version 2.13.0`

Wayfinder will output the YAML configuration to the console for further editing.

  • For this example, we'll use the default values provided in the eck-operator Helm chart. Remove the valuesTemplate section and provide a value for installNamespace. Save the modified YAML to a file named eck-operator.yaml. Your file should look similar to this:
apiVersion: package.appvia.io/v2beta2
kind: Package
metadata:
  name: eck-operator
spec:
  helm:
    chartName: eck-operator
    chartVersion: 2.13.0
    releaseName: eck-operator
    repositoryRef: elastic-stack
    values: {}
  installNamespace: elastic-system
  selectors: {}
  version: 1.0.0  #this is the version of your Wayfinder package

  • Apply the YAML configuration using the wf apply command. Ensure the file is in your working directory, or specify the full path:
wf apply -f eck-operator.yaml


3. Create a Wayfinder Package for the ECK Stack​

In this section, we will create the Wayfinder package to deploy the ECK Stack.

  • Use the wf init package PACKAGE --helm-repo REPO --chart CHART --version VERSION command to create the package YAML for the ECK Stack:
wf init package eck-stack --helm-repo https://helm.elastic.co --chart eck-stack --version 0.11.0

Wayfinder will output the YAML configuration to the console for further editing.

  • In this example, we only need the Elasticsearch and Kibana tools, so we'll override the default chart values in the valuesTemplate section. Additionally, provide a value for installNamespace. Save the modified YAML to a file named eck-stack.yaml. Your file should look similar to this:
apiVersion: package.appvia.io/v2beta2
kind: Package
metadata:
  name: eck-stack
spec:
  description: https://helm.elastic.co
  helm:
    chartName: eck-stack
    chartVersion: 0.11.0
    releaseName: eck-stack
    repositoryRef: elastic-stack
    valuesTemplate: >
      ---
      eck-elasticsearch:
        enabled: true
        fullnameOverride: elasticsearch

      eck-kibana:
        enabled: true
        spec:
          elasticsearchRef:
            name: elasticsearch

      eck-agent:
        enabled: false

      eck-fleet-server:
        enabled: false

      eck-beats:
        enabled: false

      eck-logstash:
        enabled: false

      eck-apm-server:
        enabled: false

      eck-enterprise-search:
        enabled: false
  installNamespace: eck-stack 
  selectors: {}
  version: 0.11.0-1 # this is the version of the Wayfinder package

  • Apply the YAML configuration using the wf apply command. Ensure the file is in your working directory, or specify the full path:
wf apply -f eck-stack.yaml


Validate​

4. Validate your packages​

We recommend always validating your packages against a test cluster using the wf validate package command.

For detailed instructions and example outputs, refer to the Validate the Package's Configuration section.



Include​

5. Add the packages to a cluster plan​

Instead of deploying packages directly to a cluster, add them to a new or existing cluster plan.

Below is a snippet of the cluster plan's YAML:

packages:
  - name: eck-stack
    version: 0.11.0-1
  - name: eck-operator
    version: 1.0.0

For detailed instructions, refer to the How do I add a Package to a Cluster Plan? article.



Deploy​

6. Create an Application Environment Using the Cluster Plan​

When you create an Application Environment using a Cluster Plan, Wayfinder will create a cluster (new infrastructure) based on the specifications in the cluster plan and deploy the specified packages.

For detailed instructions, refer to the How do I Create an Environment using New Infrastructure? article.