Configuring SEAL Elastic Stack via Script¶

Elasticsearch uses different indices for the log, statistics, audit and accounting data of PLOSSYS 5.

For easyPRIMA you only need to configure audit data in Kibana.

For SEAL Operator you only need to configure log and audit data in Kibana.

For the different indices you can configure the housekeeping in Elasticsearch (Index Lifecycle Management, ILM).

If you prefer to proceed manually in the Kibana user interface, you will find the step-by-step instructions in Configuring SEAL Elastic Stack by Hand.

The `load-config` Script¶

SEAL Elastic Stack provides the load-config script and several configuration files to create all necessary components for the housekeeping of the stored data of PLOSSYS 5, PLOSSYS 4, SEAL Operator and easyPRIMA.

After the version update you have to start the load-config script again to update the configuration, too.

You will find the script in the following directory:

/opt/seal/seal-kibana/configuration/load-config.sh

You may use the load-config script in different operation modes:

Non-overwrite mode:

The load-config script checks, whether the specific index exists and adds only settings that are missing.
Overwrite mode:

The load-config script overwrites some settings.

Hint - dashboards and index templates

Independent of the operation mode, the load-config script always overwrites

dashboard configurations to protect the consistency of the consecutive configurations and
index templates.

Likewise the index templates are always overwritten.

Hint - idex lifecycle policies

Independent of the operation mode, the load-config script never overwrites the index lifecycle policies.

Hint - pipelines

Pipelines, which have custom in their names, are created, but never overwritten. If you whish to have them updated, you have to delete them manually in the Kibana user interface before you start the version update.

Any non-customer-specific pipelines are always overwritten.

For details on pipelines, see Configuring pipelines.

Hint - usage

Execute the script with the -h or -help option to get the usage.

Executing the Script¶

Stop the seal-filebeat services on all PLOSSYS 5, PLOSSYS 4, SEAL Operator and easyPRIMA servers to stop the data transfer to Elasticsearch:
```
sudo systemctl stop seal-filebeat
```
```
sudo systemctl stop seal-p4-accounting-filebeat
```
```
sudo systemctl stop seal-operator-filebeat
```
By default, the configuration matches the installation of PLOSSYS 5, PLOSSYS 4, SEAL Operator and easyPRIMA. If you still need to modify the configuration, do this on the management server in the directory structure described in Configuring the Script below.
On the management server, start the load-config script for PLOSSYS 5:
```
/opt/seal/seal-kibana/configuration/load-config.sh
```

Start the load-config script a second time for easyPRIMA, if required:

/opt/seal/seal-kibana/configuration/load-config.sh -c seal-easyprima

Start the load-config script a third time for SEAL Operator, if required:
```
/opt/seal/seal-kibana/configuration/load-config.sh -c seal-operator
```
Start the load-config script once more to configure a common index pattern for SEAL, if required:
```
/opt/seal/seal-kibana/configuration/load-config.sh -c seal-common
```
You can also manually create a common index pattern for the audit, as described in Creating an Index Pattern for the Audit Indices.
Start the seal-filebeat services on all PLOSSYS 5 and management servers:
```
sudo systemctl start seal-filebeat
```
Start the seal-p4-accounting-filebeat services on the PLOSSYS 4 server, if required:
```
sudo systemctl start seal-p4-accounting-filebeat
```
Start the seal-operator-filebeat services on the SEAL Operator server, if required:
```
sudo systemctl start seal-operator-filebeat
```

Configuring the Script¶

The load-config script scans the directories stated below for JSON files and uses their content for the configuration of the different components of SEAL Elastic Stack.

If a JSON file or a subdirectory does not exist, the load-config script skips the configuration of the corresponding component:

/opt/seal/seal-kibana/configuration/<product_name>

Example - configuration directory contained in SEAL Elastic Stack 7.16.0.53

Configuration Directory of Kibana

The subdirectories in the product-specific directories contain JSON files for the different components:

index:

Elasticsearch index, for details on the syntax, see original documentation.
index-lifecycle-policy:

Housekeeping of the data in the specific index, for details on the syntax, see original documentation.
index-pattern:

Index patterns for accessing the data stored in the specific index, for details on the syntax, see original documentation.
index-template:

Template used when creating the index, for details on the syntax, see original documentation.
index-template/components:

Reusable subcomponents for index templates, for details on the syntax, see original documentation.

The JSON files in the component-specific directories are named according to the data type that is to be configured:

accounting.json
audit.json
log.json
statistics.json

The following subdirectory in the product-specific directories contains JSON files used to manipulate fields in index patterns:

fields:

Customization of fields of an index pattern, for details on the syntax, see original documentation.

The following subdirectory in the product-specific directories contains JSON files that are used to define ingest pipelines:

pipelines:

Ingest pipelines to adjust incoming data, for details on the syntax, see original documentation.

In Elasticsearch, you will find a number of predefined processors for ingest pipelines, which are comparable to filebeat processors. For details on this, see original documentation.

SEAL Elastic Stack 7.17.3 provides a concept for safely updating ingest pipelines. Aim is to combine SEAL-specific and customer-specific pipelines in a way that allows the SEAL-pipelines to be updated without overwriting the customer-specific parts.

Mit dieser Version 7.17.3 wird ein Konzept für Update-sichere Pipelines umgesetzt. Das Ziel ist, SEAL-spezifische und Customerspezifische Pipelines so zu kombinieren, dass bei einem Update die SEAL-Pipelines aktualisiert werden können, ohne die Kundenanteile zu überschreiben.

The following subdirectory in the product-specific directories contains JSON files used to adjust sample dashboards of Kibana:

dashboard:

Dashboards to visualize the log data of PLOSSYS 5, for details on the syntax, see original documentation.

The following subdirectory in the product-specific directories contains JSON files used to customize sample Kibana searches:

search:

Predefined searches for an easier analyzation of the log data, for details on the syntax, see original documentation.

The following subdirectories in the product-specific directories contain JSON files that are used to customize the Kibana user interface:

workspace:

You can use Kibana spaces to restrict the Kibana user interface to essential features, e. g. a space for working with accounting data. In the screenshot, you can see that for the SEAL Accounting space only 4 of 23 possible features are visible. you will find a deactivated sample configuration for this space in the following directory:

seal-kibana/configuration/seal-plossys-5/workspace/seal-accounting.json.example.

Best practice here is combining a Kibana space with the definition of a corresponding role and its link, e. g. with an accounting user. For details on the syntax, see original documentation.

copy-saved-objects:

When using Kibana spaces, you have 2 possibilities to set up kibana objects, like index patterns or dashboards:

- Create the kibana objects by using the `-workspace` option of the configuration script.

- Copy the kibana objects by specifying a copy statement according to the Elastic API. For details on this, see [original documentation](https://www.elastic.co/guide/en/kibana/current/spaces-api-copy-saved-objects.html).

    In this directory, you can create a JSON file for each copy statement.

Example - using different spaces

Configuration Spaces of Kibana

The following subdirectories in the product-specific directories contain JSON files that are used to automatically generate users, roles, and role mappings. The directories currently contain *.json.example files as possible examples:

security/role:

Definition of roles, for details on the syntax, see original documentation.
security/user:

Definition of users, for details on the syntax, see original documentation.

You may use an online tool for creating passwords encrypted with BCRYPT10, which is the standard of Elasticsearch, e.g. Bcrypt Hash Generator & Verifier.
security/role-mapping:

Definition of role mappings, for details on the syntax, see original documentation.

Backup Files¶

Before changing existing settings, the load-config script saves the corresponding original files in the following directory:

$HOME/tmp/backup

Log File of the Script¶

The load-config script logs its own error messages in a log file in the following directory:

$HOME/tmp/log

Indices for PLOSSYS 5¶

In some cases you need to adjust PLOSSYS 5 environment variables, e. g. if you use a Kibana workspace.

In the PLOSSYS 5 system, the indices used in Elasticsearch for the log and statistics data of PLOSSYS 5 are specified in the following keys:

ELASTICSEARCH_INDEX_LOG
ELASTICSEARCH_INDEX_STATISTICS

For details on the environment variables, refer to PLOSSYS 5.