Helm Chart for Discovery in Symphony

This applies to: Visual Data Discovery

This Helm chart is publicly available in the Symphony Helm Chart repository. Optional components are disabled by default.

This topic covers:

• Working With Helm

  • Obtain the Chart

  • Customize the Chart Before Installing

• Configuring the Chart

  • The Default Configuration

  • Deciding on the Configuration

  • Injecting Symphony Configuration Properties

    • Application Properties

      • Regular Application Properties

      • Sensitive Application Properties

      • Enable the Data Gateway

      • List of Properties Available as Helm Parameters

    • JVM Properties

      • Heap Size Configuration

      • Passing Arbitrary Java Options to Services

  • Injecting Credentials

Working With Helm

This section explains the generic usage of Helm and Helm chart lifecycle applied to the Symphony Helm Chart. For a generic introduction to Helm, see the Helm Quickstart Guide.

Obtain the Chart

Add the chart repository:

helm repo add composer https://composer-repo.logianalytics.com/helm-charts/stable

List the charts available:

• helm repo update composer

• helm search repo composer

Note:  These instructions are valid for Symphony 23.1 and later, chart version 1.5.2 and later.

Customize the Chart Before Installing

Run the following command to list all the configurable parameters of the Symphony chart and their default values:

helm show values composer/composer

Review the list of parameters to decide if anything needs to be changed. Use the guidance provided in Deciding on the Configuration to evaluate the parameters to help you decide if anything has to be changed.

Configuring the Chart

The Symphony platform has lots of configuration options and some optional components. While the out-of-the-box configuration of the Helm chart provides some meaningful defaults, most likely you'll need to customize some important aspects like the list of required connectors or the need for horizontal autoscaling.

The Default Configuration

The default configuration installs a release with the following components:

Enabled Symphony services:

• Mandatory: Web Server, Query Engine, Consul.

• Connectors: PostgreSQL.

• Optional: Data Writer.

• PostgreSQL metadata store location: internal.

Advanced capabilities:

• Default Ingress rule: ENABLED

• Tracing infrastructure: DISABLED

• Horizontal autoscaling: DISABLED

• Data Gateway: DISABLED

See the next section to learn how to customize it.

Deciding on the Configuration

This list covers the main decisions and recommended actions that you’ll need to perform to determine your configuration:

1. What is the list of data stores that you need to connect to? By default, only PostgreSQL connector is installed. Customize the list of connectors according to your needs.

2. Do you want to enable horizontal autoscaling?

3. Are you going to schedule Dashboard Reports that will deliver you dashboard screenshots periodically? If yes, enable the Screenshot Service component.

4. Are you going to upload flat files (e.g. CSV, JSON) for further analysis? If not, disable the Data Writer component to save cluster resources.

5. Do you have a license that you want to apply to this deployment? If yes, inject the license during the chart installation.

6. How do you want to integrate your software into your observability infrastructure?

Once you decide on your target configuration, prepare corresponding override values for the Helm chart and put them into your values.yaml. Override values fall into two categories:

1. Symphony application configuration properties injected into services running inside pods/containers:

   • Regular application properties

   • Credentials

2. Configuration for Kubernetes resources governed by Helm.

The next section covers Symphony application configuration.

Injecting Symphony Configuration Properties

Application Properties

There are two categories of Symphony application configuration properties that differ in their sensitivity, hence the way they are specified in the values.yaml:

• Regular application configuration properties that don’t contain sensitive data and are not exposed as separate Helm chart parameters.

• Sensitive application configuration properties, such as database credentials, that are exposed as Helm chart parameters.

Regular Application Properties

Regular properties for a Symphony service are specified as the properties map within the object representing this service in the values.yaml file. Each key-value pair in this map represents a property name and value.

For example, the following snippet shows a number of regular properties with names starting with mail. that specify mail server configuration for Zoomdata Web component:

Regular properties go to ConfigMaps when Symphony is installed in a Kubernetes cluster and are treated as if they are specified in regular properties files. The names of the properties should have the same names as in regular properties files (usually found under /etc/zoomdata or /opt/zoomdata/conf for Linux-based deployments).

Sensitive Application Properties

Each sensitive property for a Symphony service is specified as a separate parameter in the object representing this service in the values.yaml l file. For example, parameters zoomdataWeb.mailLogin and zoomdataWeb.mailPassword below are sensitive properties that specify, correspondingly, login and password for the mail server configured for Zoomdata Web component:

It is not possible to override sensitive configuration properties with regular ones. For example, in the snippet below, regular properties mail.loginand mail.password will be ignored by the Zoomdata Web component:

Sensitive properties go to Secrets when Symphony is installed in a Kubernetes cluster. They are usually injected into corresponding services through environment variables.

Enable the Data Gateway

Use a data gateway in your environment to connect to information securely outside of your environment. Use a gateway client to authenticate your connection and make the data available to users. To enable the data gateway, add these values to the values.yaml.

List of Properties Available as Helm Parameters

JVM Properties

There are two categories of properties available for all services:

• Properties to configure Symphony services heap size.

• A catch-all property that allows passing arbitrary Java options to each service.

Heap Size Configuration

The following Helm chart parameters are used to control heap size for Symphony Java services:

The following snippet shows the default heap size configuration for connectors:

The following snippet shows how to override the default settings for the PostgreSQL connector:

Passing Arbitrary Java Options to Services

A catch-all property for passing arbitrary Java options is called additionalJavaOpts and is supported for each Composer service. For example, this is how to enable garbage collector logging for Query Engine:

queryEngine: additionalJavaOpts: "-verbose:gc"

Injecting Credentials

Some services might need additional credentials provided in separate files, like Java trust stores and Kerberos keytab files. To inject such credentials, you need to use additional Kerberos Secrets and Volumes.

For example, let’s consider how to inject a Java trust store into the Elasticsearch 8 connector:

1. Create a trust store Secret:

   kubectl create secret generic <secret-name> --from-file=truststore.p12

2. Configure the Elasticsearch 8 connector to mount this Secret as a volume and access the trust store from its file system:

   edc: elasticsearch-8.0: enabled: true additionalJavaOpts: "-Djavax.net.ssl.trustStore=/opt/zoomdata/security/truststore.p12 -Djavax.net.ssl.truststoreType=PKCS12 -Djavax.net.ssl.trustStorePassword=<truststore-password>" extraVolumeMounts: - name: truststore mountPath: /opt/zoomdata/security/truststore.p12 subPath: truststore.p12 readOnly: true extraVolumes: - name: truststore secret: secretName: <secret-name>

3. Install or upgrade the Helm chart.