Helm Chart for Discovery in Symphony
This applies to: Visual Data Discovery
This Helm chart is publicly available in the Symphony Helm Chart repository. Optional components are disabled by default.
This topic covers:
Working With Helm
This section explains the generic usage of Helm and Helm chart lifecycle applied to the Symphony Helm Chart. For a generic introduction to Helm, see the Helm Quickstart Guide.
Obtain the Chart
Add the chart repository:
helm repo add composer https://composer-repo.logianalytics.com/helm-charts/stable
List the charts available:
helm repo update composerhelm search repo composer
Note: These instructions are valid for Symphony 23.1 and later, chart version 1.5.2 and later.
Customize the Chart Before Installing
Run the following command to list all the configurable parameters of the Symphony chart and their default values:
helm show values composer/composer
Review the list of parameters to decide if anything needs to be changed. Use the guidance provided in Deciding on the Configuration to evaluate the parameters to help you decide if anything has to be changed.
Configuring the Chart
The Symphony platform has lots of configuration options and some optional components. While the out-of-the-box configuration of the Helm chart provides some meaningful defaults, most likely you'll need to customize some important aspects like the list of required connectors or the need for horizontal autoscaling.
The Default Configuration
The default configuration installs a release with the following components:
Enabled Symphony services:
Mandatory: Web Server, Query Engine, Consul.
Connectors: PostgreSQL.
Optional: Data Writer.
PostgreSQL metadata store location: internal.
Advanced capabilities:
Default Ingress rule:
ENABLEDTracing infrastructure:
DISABLEDHorizontal autoscaling:
DISABLEDData Gateway:
DISABLED
See the next section to learn how to customize it.
Deciding on the Configuration
This list covers the main decisions and recommended actions that you’ll need to perform to determine your configuration:
What is the list of data stores that you need to connect to? By default, only PostgreSQL connector is installed. Customize the list of connectors according to your needs.
Do you want to enable horizontal autoscaling?
Are you going to schedule Dashboard Reports that will deliver you dashboard screenshots periodically? If yes, enable the Screenshot Service component.
Are you going to upload flat files (e.g. CSV, JSON) for further analysis? If not, disable the Data Writer component to save cluster resources.
Do you have a license that you want to apply to this deployment? If yes, inject the license during the chart installation.
How do you want to integrate your software into your observability infrastructure?
Once you decide on your target configuration, prepare corresponding override values for the Helm chart and put them into your values.yaml. Override values fall into two categories:
-
Symphony application configuration properties injected into services running inside pods/containers:
Regular application properties
Credentials
Configuration for Kubernetes resources governed by Helm.
The next section covers Symphony application configuration.
Injecting Symphony Configuration Properties
Application Properties
There are two categories of Symphony application configuration properties that differ in their sensitivity, hence the way they are specified in the values.yaml:
Regular application configuration properties that don’t contain sensitive data and are not exposed as separate Helm chart parameters.
Sensitive application configuration properties, such as database credentials, that are exposed as Helm chart parameters.
Regular Application Properties
Regular properties for a Symphony service are specified as the properties map within the object representing this service in the values.yaml file. Each key-value pair in this map represents a property name and value.
For example, the following snippet shows a number of regular properties with names starting with mail. that specify mail server configuration for Zoomdata Web component:
Regular properties go to ConfigMaps when Symphony is installed in a Kubernetes cluster and are treated as if they are specified in regular properties files. The names of the properties should have the same names as in regular properties files (usually found under /etc/zoomdata or /opt/zoomdata/conf for Linux-based deployments).
Sensitive Application Properties
Each sensitive property for a Symphony service is specified as a separate parameter in the object representing this service in the values.yaml l file. For example, parameters zoomdataWeb.mailLogin and zoomdataWeb.mailPassword below are sensitive properties that specify, correspondingly, login and password for the mail server configured for Zoomdata Web component:
It is not possible to override sensitive configuration properties with regular ones. For example, in the snippet below, regular properties mail.loginand mail.password will be ignored by the Zoomdata Web component:
Sensitive properties go to Secrets when Symphony is installed in a Kubernetes cluster. They are usually injected into corresponding services through environment variables.
Enable the Data Gateway
Use a data gateway in your environment to connect to information securely outside of your environment. Use a gateway client to authenticate your connection and make the data available to users. To enable the data gateway, add these values to the values.yaml.
List of Properties Available as Helm Parameters
| Parameter in Values File | Description | Injected As | Corresponding Application Property |
|---|---|---|---|
Zoomdata Web | |||
|
Metadata database URL |
Env variable |
|
|
Metadata database username |
Secret |
|
|
Metadata database password |
Secret |
|
|
Upload database URL |
Env variable |
|
|
Upload database username |
Secret |
|
|
Upload database password |
Secret |
|
|
Keyset database URL |
Env variable |
|
|
Keyset database username |
Secret |
|
|
Keyset database password |
Secret |
|
|
User auditing database URL |
Env variable |
|
|
User auditing database username |
Secret |
|
|
User auditing database password |
Secret |
|
|
Mail server login |
Secret |
|
|
Mail server password |
Secret |
|
|
Password for the built-in |
Secret |
|
Query Engine | |||
|
Use Query Engine database for storing query results cache when |
|
|
|
Query Engine database URL |
Env variable |
|
|
Query Engine database username |
Secret |
|
|
Query Engine database password |
Secret |
|
JVM Properties
There are two categories of properties available for all services:
Properties to configure Symphony services heap size.
A catch-all property that allows passing arbitrary Java options to each service.
Heap Size Configuration
The following Helm chart parameters are used to control heap size for Symphony Java services:
Property in Values File |
Description |
Scope |
Java Option |
|---|---|---|---|
|
Initial JVM heap size. |
Service |
|
|
Maximum JVM heap size. |
Service |
|
The following snippet shows the default heap size configuration for connectors:
The following snippet shows how to override the default settings for the PostgreSQL connector:
Passing Arbitrary Java Options to Services
A catch-all property for passing arbitrary Java options is called additionalJavaOpts and is supported for each Composer service. For example, this is how to enable garbage collector logging for Query Engine:
queryEngine: additionalJavaOpts: "-verbose:gc"
Injecting Credentials
Some services might need additional credentials provided in separate files, like Java trust stores and Kerberos keytab files. To inject such credentials, you need to use additional Kerberos Secrets and Volumes.
For example, let’s consider how to inject a Java trust store into the Elasticsearch 8 connector:
-
Create a trust store Secret:
kubectl create secret generic <secret-name> --from-file=truststore.p12
-
Configure the Elasticsearch 8 connector to mount this Secret as a volume and access the trust store from its file system:
edc: elasticsearch-8.0: enabled: true additionalJavaOpts: "-Djavax.net.ssl.trustStore=/opt/zoomdata/security/truststore.p12 -Djavax.net.ssl.truststoreType=PKCS12 -Djavax.net.ssl.trustStorePassword=<truststore-password>" extraVolumeMounts: - name: truststore mountPath: /opt/zoomdata/security/truststore.p12 subPath: truststore.p12 readOnly: true extraVolumes: - name: truststore secret: secretName: <secret-name>
Comments
0 comments
Please sign in to leave a comment.