Indexes customizations and management

Default indexes model

The application stack uses by default a 4 layer indexes models:

Index name Purpose
os-unix-nmon-metrics metric store index where metrics data is being stored
os-unix-nmon-events nmon data stored as regular events
os-unix-nmon-config nmon configuration data
os-unix-nmon-internal internal data generated by the application for various processing tasks


The indexes are defined in the Support Addon SA-metricator-for-nmon:



The front-end application metricator-for-nmon uses macro definitions for the indexes to be used by the application:


List of indexes macros:

macro name indexes target
nmon_index index=os-unix-nmon-*
nmon_idx idx=os-unix-nmon-*
nmon_events_index index=os-unix-nmon-events*
nmon_metrics_index index=os-unix-nmon-metrics*

Wildcards are automatically replaced by any suite of characters.


The front-end application metricator-for-nmon uses events definition in eventtypes.conf:

eventtype indexes target
nmon index=os-unix-nmon-events* OR index=os-unix-nmon-internal*
nmon:events index=os-unix-nmon-events* sourcetype=nmon_data
nmon:config index=os-unix-nmon-config* sourcetype=nmon_config
nmon:collect index=os-unix-nmon-internal* sourcetype=nmon_collect
nmon:processing index=os-unix-nmon-internal* sourcetype=nmon_processing
nmon:clean index=os-unix-nmon-internal* sourcetype=nmon_clean
uptime index=os-unix-nmon-events* sourcetype=nmon_data type=UPTIME
inventory index=os-unix-nmon-config* sourcetype=nmon_config


The Technical Addon contains a default definition of indexes to be used for data indexing:


With following inputs stanza specifying their target index:

index = os-unix-nmon-internal

index = os-unix-nmon-internal

index = os-unix-nmon-metrics

index = os-unix-nmon-events

index = os-unix-nmon-config

[script://./bin/ --cleancsv]
index = os-unix-nmon-internal

Customizing indexes names and multi-tenant configuration

For various reasons and purposes, you may need to customize the indexes names:

  • You have your own naming convention and you wish to be able to use for the application deployment as well
  • You are a large company with multiple regions across the globe and you want a geographical or per environment segregation
  • You want to use access control based indexes segregation to allow specific populations to access specific indexes
  • Any combination of these

The applications stack is compatible with any kind of customization.

Recommended customization scenario:

As we use a naming convention which matches indexes using wildcards by default, you are free to use any convention starting by the index names, such as:

Default Index name Automatic target
os-unix-nmon-metrics os-unix-nmon-metrics*
os-unix-nmon-events os-unix-nmon-events*
os-unix-nmon-config os-unix-nmon-config*
os-unix-nmon-internal os-unix-nmon-internal*

Using our naming convention

If you are using our naming built-in naming convention then good news for you, you have absolutely nothing else to do than creating your own indexes.

Example of configuration: per environment segregation

Let’s take the example of having 3 environments where you are deploying the application stack:

  • Engineering (-eng)
  • UAT (-uat)
  • Production (-prod)

Because you may want to use a much higher retention for Production systems rather than Eng and UAT and have a larger period of metrics available, this is valuable to split the ingestion into multiple-indexes stacks.

You proceed the following way:

  • For each of your indexer layer, prepare the stack of indexes and apply your custom indexes
  • Create a local version of the inputs.conf shipped with the Technical Addon and publish it to the instances managed by the deployment servers
  • Respect Splunk best practices in every step!

Based on the example above::

Indexing layer preparation:

Production configuration, assuming clustered indexers.

On cluster master:

cd /opt/splunk/etc/master-apps/SA-metricator-for-nmon
if [ ! -d local ]; then mkdir local; fi
cp -p default/indexes.conf local/

Then achieve the following configuration, adapt if using Splunk volumes or for any other need:

# nmon data ingested as metrics
disabled = false
coldPath = $SPLUNK_DB/os-unix-nmon-metrics-prod/colddb
datatype = metric
homePath = $SPLUNK_DB/os-unix-nmon-metrics-prod/db
splitByIndexKeys = metric_name,host
thawedPath = $SPLUNK_DB/os-unix-nmon-metrics-prod/thaweddb
repFactor = auto

# nmon data ingested as regular events
disabled = false
coldPath = $SPLUNK_DB/os-unix-nmon-events-prod/colddb
homePath = $SPLUNK_DB/os-unix-nmon-events-prod/db
thawedPath = $SPLUNK_DB/os-unix-nmon-events-prod/thaweddb
repFactor = auto

# nmon config ingested as regular events
disabled = false
coldPath = $SPLUNK_DB/os-unix-nmon-config-prod/colddb
homePath = $SPLUNK_DB/os-unix-nmon-config-prod/db
thawedPath = $SPLUNK_DB/os-unix-nmon-config-prod/thaweddb
repFactor = auto

# nmon internal data
disabled = false
coldPath = $SPLUNK_DB/os-unix-nmon-internal-prod/colddb
homePath = $SPLUNK_DB/os-unix-nmon-internal-prod/db
thawedPath = $SPLUNK_DB/os-unix-nmon-internal-prod/thaweddb
repFactor = auto

Publish the cluster bundle

Technical Addon configuration

On Deployment Servers:

cd /opt/splunk/etc/deployment-apps/TA-metricator-for-nmon
if [ ! -d local ]; then mkdir local; fi

Add the following configuration to local/inputs.conf (create the file)

index = os-unix-nmon-internal-prod

index = os-unix-nmon-internal-prod

index = os-unix-nmon-metrics-prod

index = os-unix-nmon-events-prod

index = os-unix-nmon-config-prod

[script://./bin/ --cleancsv]
index = os-unix-nmon-internal-prod

Achieve the same configuration on any server running the TA-metricator-for-nmon and not managed by Deployment Servers (cluster master, Deployment Servers…)

Reload or restart the Deployment Servers to publish this new configuration to managed servers

And finally repeat the operation for each environment that needs to be managed.

It is not required to achieve any modification on the front-end user application since the default configuration takes in charge any index respecting the naming convention using wildcards definitions.

Using a different naming convention

If the builtin indexes naming convention is not option for you, it is still possible to customize the different applications to match your needs.

However for easy of use and maintenance, we would recommend using the builtin naming convention when possible to avoid additional steps in your deployments.

You will proceed as following:

  • Customize indexes in local/indexes.conf (Indexer layer in SA-metricator-for-nmon)
  • Customize macros in local/macros.conf (Search Head layer in metricator-for-nmon)
  • Customize eventtypes in local/eventtypes.conf (Search Head layer in metricator-for-nmon)
  • Customize inputs.conf in local/inputs.conf (Technical Addon in TA-metricator-for-nmon)