docs(aggregators): Add plugin metadata and update description (#16089)

This commit is contained in:
Sven Rebhan 2024-10-30 15:29:55 +01:00 committed by GitHub
parent 6045e13616
commit d76723d21c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 107 additions and 86 deletions

View File

@ -1,8 +1,11 @@
# BasicStats Aggregator Plugin # Basic Statistics Aggregator Plugin
The BasicStats aggregator plugin gives count, diff, max, min, mean, This plugin computes basic statistics such as counts, differences, minima,
non_negative_diff, sum, s2(variance), stdev for a set of values, last and maxima, mean values, non-negative differences etc. for a set of metrics and
first, emitting the aggregate every `period` seconds. emits these statistical values every `period`.
⭐ Telegraf v1.5.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->

View File

@ -1,7 +1,9 @@
# Derivative Aggregator Plugin # Derivative Aggregator Plugin
The Derivative Aggregator Plugin estimates the derivative for all fields of the This plugin computes the derivative for all fields of the aggregated metrics.
aggregated metrics.
⭐ Telegraf v1.18.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->

View File

@ -1,15 +1,24 @@
# Final Aggregator Plugin # Final Aggregator Plugin
The final aggregator emits the last metric of a contiguous series. A This plugin emits the last metric of a contiguous series, defined as a
contiguous series is defined as a series which receives updates within the series which receives updates within the time period in `series_timeout`. The
time period in `series_timeout`. The contiguous series may be longer than the contiguous series may be longer than the time interval defined by `period`.
time interval defined by `period`. When a series has not been updated within the `series_timeout`, the last metric
is emitted.
Alternatively, the plugin emits the last metric in the `period` for the
`periodic` output strategy.
This is useful for getting the final value for data sources that produce This is useful for getting the final value for data sources that produce
discrete time series such as procstat, cgroup, kubernetes etc. discrete time series such as procstat, cgroup, kubernetes etc. or to downsample
metrics collected at a higher frequency.
When a series has not been updated within the time defined in > [!NOTE]
`series_timeout`, the last metric is emitted with the `_final` appended. > All emited metrics do have fields with `_final` appended to the field-name
> by default.
⭐ Telegraf v1.11.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->

View File

@ -1,33 +1,21 @@
# Histogram Aggregator Plugin # Histogram Aggregator Plugin
The histogram aggregator plugin creates histograms containing the counts of This plugin creates histograms containing the counts of field values within the
field values within a range. configured range. The histogram metric is emitted every `period`.
If `cumulative` is set to true, values added to a bucket are also added to the In `cumulative` mode, values added to a bucket are also added to the
larger buckets in the distribution. This creates a [cumulative histogram][1]. consecutive buckets in the distribution creating a [cumulative histogram][1].
Otherwise, values are added to only one bucket, which creates an [ordinary
histogram][1]
Like other Telegraf aggregators, the metric is emitted every `period` seconds. > [!NOTE]
By default bucket counts are not reset between periods and will be non-strictly > By default bucket counts are not reset between periods and will be
increasing while Telegraf is running. This behavior can be changed by setting > non-strictly increasing while Telegraf is running. This behavior can be
the `reset` parameter to true. > by setting the `reset` parameter.
⭐ Telegraf v1.4.0
💻 all
[1]: https://en.wikipedia.org/wiki/Histogram#/media/File:Cumulative_vs_normal_histogram.svg [1]: https://en.wikipedia.org/wiki/Histogram#/media/File:Cumulative_vs_normal_histogram.svg
## Design
Each metric is passed to the aggregator and this aggregator searches histogram
buckets for those fields, which have been specified in the config. If buckets
are found, the aggregator will increment +1 to the appropriate
bucket. Otherwise, it will be added to the `+Inf` bucket. Every `period`
seconds this data will be forwarded to the outputs.
The algorithm of hit counting to buckets was implemented on the base of the
algorithm which is implemented in the Prometheus [client][2].
[2]: https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->
In addition to the plugin-specific configuration settings, plugins support In addition to the plugin-specific configuration settings, plugins support

View File

@ -1,11 +1,14 @@
# Merge Aggregator Plugin # Merge Aggregator Plugin
Merge metrics together into a metric with multiple fields into the most memory This plugin merges metrics of the same series and timestamp into new metrics
and network transfer efficient form. with the super-set of fields. A series here is defined by the metric name and
the tag key-value set.
Use this plugin when fields are split over multiple metrics, with the same Use this plugin when fields are split over multiple metrics, with the same
measurement, tag set and timestamp. By merging into a single metric they can measurement, tag set and timestamp.
be handled more efficiently by the output.
⭐ Telegraf v1.13.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->

View File

@ -1,7 +1,11 @@
# MinMax Aggregator Plugin # Minimum-Maximum Aggregator Plugin
The minmax aggregator plugin aggregates min & max values of each field it sees, This plugin aggregates the minimum and maximum values of each field it sees,
emitting the aggrate every `period` seconds. emitting the aggrate every `period` seconds with field names suffixed by `_min`
and `_max` respectively.
⭐ Telegraf v1.1.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->

View File

@ -1,7 +1,11 @@
# Quantile Aggregator Plugin # Quantile Aggregator Plugin
The quantile aggregator plugin aggregates specified quantiles for each numeric This plugin aggregates each numeric field per metric into the specified
field per metric it sees and emits the quantiles every `period`. quantiles and emits the quantiles every `period`. Different aggregation
algorithms are supported with varying accuracy and limitations.
⭐ Telegraf v1.18.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->
@ -135,11 +139,6 @@ cpu,cpu=cpu-total,host=Hugin usage_idle=85.78199052131747,usage_nice=0,usage_irq
cpu,cpu=cpu-total,host=Hugin usage_guest_nice_075=0,usage_user_050=10.814851731872487,usage_guest_075=0,usage_steal_025=0,usage_irq_025=1.031558489546918,usage_irq_075=1.0471206791944527,usage_iowait_025=0,usage_guest_050=0,usage_guest_nice_050=0,usage_nice_075=0,usage_iowait_050=0,usage_system_050=2.1601016518428664,usage_irq_050=1.046598554697342,usage_guest_nice_025=0,usage_idle_050=85.79616247197244,usage_softirq_075=0.1887208672481664,usage_steal_075=0,usage_system_025=2.0778058770562287,usage_system_075=2.1640279004292173,usage_softirq_050=0.1778907242693666,usage_nice_050=0,usage_iowait_075=0.01270648030495347,usage_user_075=10.895078647178611,usage_nice_025=0,usage_steal_050=0,usage_user_025=10.04529117724472,usage_idle_025=85.78907649664495,usage_idle_075=86.57025404411868,usage_softirq_025=0.1761619083594677,usage_guest_025=0 1608288390000000000 cpu,cpu=cpu-total,host=Hugin usage_guest_nice_075=0,usage_user_050=10.814851731872487,usage_guest_075=0,usage_steal_025=0,usage_irq_025=1.031558489546918,usage_irq_075=1.0471206791944527,usage_iowait_025=0,usage_guest_050=0,usage_guest_nice_050=0,usage_nice_075=0,usage_iowait_050=0,usage_system_050=2.1601016518428664,usage_irq_050=1.046598554697342,usage_guest_nice_025=0,usage_idle_050=85.79616247197244,usage_softirq_075=0.1887208672481664,usage_steal_075=0,usage_system_025=2.0778058770562287,usage_system_075=2.1640279004292173,usage_softirq_050=0.1778907242693666,usage_nice_050=0,usage_iowait_075=0.01270648030495347,usage_user_075=10.895078647178611,usage_nice_025=0,usage_steal_050=0,usage_user_025=10.04529117724472,usage_idle_025=85.78907649664495,usage_idle_075=86.57025404411868,usage_softirq_025=0.1761619083594677,usage_guest_025=0 1608288390000000000
``` ```
## References
- Dunning & Ertl: "Computing Extremely Accurate Quantiles Using t-Digests", arXiv:1902.04023 (2019) [pdf][tdigest_paper]
- Hyndman & Fan: "Sample Quantiles in Statistical Packages", The American Statistician, vol. 50, pp. 361-365 (1996) [pdf][hyndman_fan]
[tdigest_paper]: https://arxiv.org/abs/1902.04023 [tdigest_paper]: https://arxiv.org/abs/1902.04023
[tdigest_lib]: https://github.com/caio/go-tdigest [tdigest_lib]: https://github.com/caio/go-tdigest
[hyndman_fan]: http://www.maths.usyd.edu.au/u/UG/SM/STAT3022/r/current/Misc/Sample%20Quantiles%20in%20Statistical%20Packages.pdf [hyndman_fan]: http://www.maths.usyd.edu.au/u/UG/SM/STAT3022/r/current/Misc/Sample%20Quantiles%20in%20Statistical%20Packages.pdf

View File

@ -1,26 +1,37 @@
# Starlark Aggregator Plugin # Starlark Aggregator Plugin
The `starlark` aggregator allows to implement a custom aggregator plugin with a This plugin allows to implement a custom aggregator plugin via a
Starlark script. The Starlark script needs to be composed of the three methods [Starlark][starlark] script.
defined in the Aggregator plugin interface which are `add`, `push` and `reset`.
The Starlark Aggregator plugin calls the Starlark function `add` to add the The Starlark language is a dialect of Python and will be familiar to those who
metrics to the aggregator, then calls the Starlark function `push` to push the
resulting metrics into the accumulator and finally calls the Starlark function
`reset` to reset the entire state of the plugin.
The Starlark functions can use the global function `state` to keep temporary the
metrics to aggregate.
The Starlark language is a dialect of Python, and will be familiar to those who
have experience with the Python language. However, there are major have experience with the Python language. However, there are major
[differences](#python-differences). Existing [differences](#python-differences). Existing Python code is unlikely to work
Python code is unlikely to work unmodified. The execution environment is unmodified.
sandboxed, and it is not possible to do I/O operations such as reading from
files or sockets.
The **[Starlark specification][]** has details about the syntax and available > [!NOTE]
functions. > The execution environment is sandboxed, and it is not possible to access the
> local filesystem or perfoming network operations. This is by design of the
> Starlark language as a configuration language.
The Starlark script used by this plugin needs to be composed of the three
methods defining an aggreagtor named `add`, `push` and `reset`.
The `add` method is called as soon as a new metric is added to the plugin the
metrics to the aggregator. After `period`, the `push` method is called to
output the resulting metrics and finally the aggregation is reset by using the
`reset` method of the Starlark script.
The Starlark functions might use the global function `state` to keep aggregation
information such as added metrics etc.
More details on the syntax and available functions can be found in the
[Starlark specification][spec].
⭐ Telegraf v1.21.0
💻 all
[starlark]: https://github.com/google/starlark-go
[spec]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->
@ -95,7 +106,7 @@ def reset():
``` ```
For a list of available types and functions that can be used in the code, see For a list of available types and functions that can be used in the code, see
the [Starlark specification][]. the [Starlark specification][spec].
## Python Differences ## Python Differences
@ -117,11 +128,9 @@ documentation about the Starlark processor.
## Examples ## Examples
- [minmax](testdata/min_max.star) - A minmax aggregator implemented with a Starlark script. - [minmax](testdata/min_max.star)
- [merge](testdata/merge.star) - A merge aggregator implemented with a Starlark script. - [merge](testdata/merge.star)
[All examples](testdata) are in the testdata folder. [All examples](testdata) are in the testdata folder.
Open a Pull Request to add any other useful Starlark examples. Open a Pull Request to add any other useful Starlark examples.
[Starlark specification]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md

View File

@ -1,19 +1,23 @@
# Value Counter Aggregator Plugin # Value Counter Aggregator Plugin
The valuecounter plugin counts the occurrence of values in fields and emits the This plugin counts the occurrence of unique values in fields and emits the
counter once every 'period' seconds. counter once every `period` with the field-names being suffixed by the unique
value converted to `string`.
A use case for the valuecounter plugin is when you are processing a HTTP access > [!NOTE]
log (with the logparser input) and want to count the HTTP status codes. > The fields to be counted must be configured using the `fields` setting,
> otherwise no field will be counted and no metric is emitted.
The fields which will be counted must be configured with the `fields` This plugin is useful to e.g. count the occurrances of HTTP status codes or
configuration directive. When no `fields` is provided the plugin will not count other categorical values in the defined `period`.
any fields. The results are emitted in fields in the format:
`originalfieldname_fieldvalue = count`.
Counting fields with a high number of potential values may produce significant > [!IMPORTANT]
amounts of new fields and memory usage, take care to only count fields with a > Counting fields with a high number of potential values may produce a
limited set of values. > significant amounts of new fields and results in an increased memory usage.
> Take care to only count fields with a limited set of values.
⭐ Telegraf v1.8.0
💻 all
## Global configuration options <!-- @/docs/includes/plugin_config.md --> ## Global configuration options <!-- @/docs/includes/plugin_config.md -->