docs(aggregators): Add plugin metadata and update description (#16089)
This commit is contained in:
parent
6045e13616
commit
d76723d21c
|
|
@ -1,8 +1,11 @@
|
||||||
# BasicStats Aggregator Plugin
|
# Basic Statistics Aggregator Plugin
|
||||||
|
|
||||||
The BasicStats aggregator plugin gives count, diff, max, min, mean,
|
This plugin computes basic statistics such as counts, differences, minima,
|
||||||
non_negative_diff, sum, s2(variance), stdev for a set of values, last and
|
maxima, mean values, non-negative differences etc. for a set of metrics and
|
||||||
first, emitting the aggregate every `period` seconds.
|
emits these statistical values every `period`.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.5.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,9 @@
|
||||||
# Derivative Aggregator Plugin
|
# Derivative Aggregator Plugin
|
||||||
|
|
||||||
The Derivative Aggregator Plugin estimates the derivative for all fields of the
|
This plugin computes the derivative for all fields of the aggregated metrics.
|
||||||
aggregated metrics.
|
|
||||||
|
⭐ Telegraf v1.18.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,15 +1,24 @@
|
||||||
# Final Aggregator Plugin
|
# Final Aggregator Plugin
|
||||||
|
|
||||||
The final aggregator emits the last metric of a contiguous series. A
|
This plugin emits the last metric of a contiguous series, defined as a
|
||||||
contiguous series is defined as a series which receives updates within the
|
series which receives updates within the time period in `series_timeout`. The
|
||||||
time period in `series_timeout`. The contiguous series may be longer than the
|
contiguous series may be longer than the time interval defined by `period`.
|
||||||
time interval defined by `period`.
|
When a series has not been updated within the `series_timeout`, the last metric
|
||||||
|
is emitted.
|
||||||
|
|
||||||
|
Alternatively, the plugin emits the last metric in the `period` for the
|
||||||
|
`periodic` output strategy.
|
||||||
|
|
||||||
This is useful for getting the final value for data sources that produce
|
This is useful for getting the final value for data sources that produce
|
||||||
discrete time series such as procstat, cgroup, kubernetes etc.
|
discrete time series such as procstat, cgroup, kubernetes etc. or to downsample
|
||||||
|
metrics collected at a higher frequency.
|
||||||
|
|
||||||
When a series has not been updated within the time defined in
|
> [!NOTE]
|
||||||
`series_timeout`, the last metric is emitted with the `_final` appended.
|
> All emited metrics do have fields with `_final` appended to the field-name
|
||||||
|
> by default.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.11.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,33 +1,21 @@
|
||||||
# Histogram Aggregator Plugin
|
# Histogram Aggregator Plugin
|
||||||
|
|
||||||
The histogram aggregator plugin creates histograms containing the counts of
|
This plugin creates histograms containing the counts of field values within the
|
||||||
field values within a range.
|
configured range. The histogram metric is emitted every `period`.
|
||||||
|
|
||||||
If `cumulative` is set to true, values added to a bucket are also added to the
|
In `cumulative` mode, values added to a bucket are also added to the
|
||||||
larger buckets in the distribution. This creates a [cumulative histogram][1].
|
consecutive buckets in the distribution creating a [cumulative histogram][1].
|
||||||
Otherwise, values are added to only one bucket, which creates an [ordinary
|
|
||||||
histogram][1]
|
|
||||||
|
|
||||||
Like other Telegraf aggregators, the metric is emitted every `period` seconds.
|
> [!NOTE]
|
||||||
By default bucket counts are not reset between periods and will be non-strictly
|
> By default bucket counts are not reset between periods and will be
|
||||||
increasing while Telegraf is running. This behavior can be changed by setting
|
> non-strictly increasing while Telegraf is running. This behavior can be
|
||||||
the `reset` parameter to true.
|
> by setting the `reset` parameter.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.4.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
[1]: https://en.wikipedia.org/wiki/Histogram#/media/File:Cumulative_vs_normal_histogram.svg
|
[1]: https://en.wikipedia.org/wiki/Histogram#/media/File:Cumulative_vs_normal_histogram.svg
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Each metric is passed to the aggregator and this aggregator searches histogram
|
|
||||||
buckets for those fields, which have been specified in the config. If buckets
|
|
||||||
are found, the aggregator will increment +1 to the appropriate
|
|
||||||
bucket. Otherwise, it will be added to the `+Inf` bucket. Every `period`
|
|
||||||
seconds this data will be forwarded to the outputs.
|
|
||||||
|
|
||||||
The algorithm of hit counting to buckets was implemented on the base of the
|
|
||||||
algorithm which is implemented in the Prometheus [client][2].
|
|
||||||
|
|
||||||
[2]: https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go
|
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
In addition to the plugin-specific configuration settings, plugins support
|
In addition to the plugin-specific configuration settings, plugins support
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,14 @@
|
||||||
# Merge Aggregator Plugin
|
# Merge Aggregator Plugin
|
||||||
|
|
||||||
Merge metrics together into a metric with multiple fields into the most memory
|
This plugin merges metrics of the same series and timestamp into new metrics
|
||||||
and network transfer efficient form.
|
with the super-set of fields. A series here is defined by the metric name and
|
||||||
|
the tag key-value set.
|
||||||
|
|
||||||
Use this plugin when fields are split over multiple metrics, with the same
|
Use this plugin when fields are split over multiple metrics, with the same
|
||||||
measurement, tag set and timestamp. By merging into a single metric they can
|
measurement, tag set and timestamp.
|
||||||
be handled more efficiently by the output.
|
|
||||||
|
⭐ Telegraf v1.13.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,11 @@
|
||||||
# MinMax Aggregator Plugin
|
# Minimum-Maximum Aggregator Plugin
|
||||||
|
|
||||||
The minmax aggregator plugin aggregates min & max values of each field it sees,
|
This plugin aggregates the minimum and maximum values of each field it sees,
|
||||||
emitting the aggrate every `period` seconds.
|
emitting the aggrate every `period` seconds with field names suffixed by `_min`
|
||||||
|
and `_max` respectively.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.1.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,11 @@
|
||||||
# Quantile Aggregator Plugin
|
# Quantile Aggregator Plugin
|
||||||
|
|
||||||
The quantile aggregator plugin aggregates specified quantiles for each numeric
|
This plugin aggregates each numeric field per metric into the specified
|
||||||
field per metric it sees and emits the quantiles every `period`.
|
quantiles and emits the quantiles every `period`. Different aggregation
|
||||||
|
algorithms are supported with varying accuracy and limitations.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.18.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
@ -135,11 +139,6 @@ cpu,cpu=cpu-total,host=Hugin usage_idle=85.78199052131747,usage_nice=0,usage_irq
|
||||||
cpu,cpu=cpu-total,host=Hugin usage_guest_nice_075=0,usage_user_050=10.814851731872487,usage_guest_075=0,usage_steal_025=0,usage_irq_025=1.031558489546918,usage_irq_075=1.0471206791944527,usage_iowait_025=0,usage_guest_050=0,usage_guest_nice_050=0,usage_nice_075=0,usage_iowait_050=0,usage_system_050=2.1601016518428664,usage_irq_050=1.046598554697342,usage_guest_nice_025=0,usage_idle_050=85.79616247197244,usage_softirq_075=0.1887208672481664,usage_steal_075=0,usage_system_025=2.0778058770562287,usage_system_075=2.1640279004292173,usage_softirq_050=0.1778907242693666,usage_nice_050=0,usage_iowait_075=0.01270648030495347,usage_user_075=10.895078647178611,usage_nice_025=0,usage_steal_050=0,usage_user_025=10.04529117724472,usage_idle_025=85.78907649664495,usage_idle_075=86.57025404411868,usage_softirq_025=0.1761619083594677,usage_guest_025=0 1608288390000000000
|
cpu,cpu=cpu-total,host=Hugin usage_guest_nice_075=0,usage_user_050=10.814851731872487,usage_guest_075=0,usage_steal_025=0,usage_irq_025=1.031558489546918,usage_irq_075=1.0471206791944527,usage_iowait_025=0,usage_guest_050=0,usage_guest_nice_050=0,usage_nice_075=0,usage_iowait_050=0,usage_system_050=2.1601016518428664,usage_irq_050=1.046598554697342,usage_guest_nice_025=0,usage_idle_050=85.79616247197244,usage_softirq_075=0.1887208672481664,usage_steal_075=0,usage_system_025=2.0778058770562287,usage_system_075=2.1640279004292173,usage_softirq_050=0.1778907242693666,usage_nice_050=0,usage_iowait_075=0.01270648030495347,usage_user_075=10.895078647178611,usage_nice_025=0,usage_steal_050=0,usage_user_025=10.04529117724472,usage_idle_025=85.78907649664495,usage_idle_075=86.57025404411868,usage_softirq_025=0.1761619083594677,usage_guest_025=0 1608288390000000000
|
||||||
```
|
```
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Dunning & Ertl: "Computing Extremely Accurate Quantiles Using t-Digests", arXiv:1902.04023 (2019) [pdf][tdigest_paper]
|
|
||||||
- Hyndman & Fan: "Sample Quantiles in Statistical Packages", The American Statistician, vol. 50, pp. 361-365 (1996) [pdf][hyndman_fan]
|
|
||||||
|
|
||||||
[tdigest_paper]: https://arxiv.org/abs/1902.04023
|
[tdigest_paper]: https://arxiv.org/abs/1902.04023
|
||||||
[tdigest_lib]: https://github.com/caio/go-tdigest
|
[tdigest_lib]: https://github.com/caio/go-tdigest
|
||||||
[hyndman_fan]: http://www.maths.usyd.edu.au/u/UG/SM/STAT3022/r/current/Misc/Sample%20Quantiles%20in%20Statistical%20Packages.pdf
|
[hyndman_fan]: http://www.maths.usyd.edu.au/u/UG/SM/STAT3022/r/current/Misc/Sample%20Quantiles%20in%20Statistical%20Packages.pdf
|
||||||
|
|
|
||||||
|
|
@ -1,26 +1,37 @@
|
||||||
# Starlark Aggregator Plugin
|
# Starlark Aggregator Plugin
|
||||||
|
|
||||||
The `starlark` aggregator allows to implement a custom aggregator plugin with a
|
This plugin allows to implement a custom aggregator plugin via a
|
||||||
Starlark script. The Starlark script needs to be composed of the three methods
|
[Starlark][starlark] script.
|
||||||
defined in the Aggregator plugin interface which are `add`, `push` and `reset`.
|
|
||||||
|
|
||||||
The Starlark Aggregator plugin calls the Starlark function `add` to add the
|
The Starlark language is a dialect of Python and will be familiar to those who
|
||||||
metrics to the aggregator, then calls the Starlark function `push` to push the
|
|
||||||
resulting metrics into the accumulator and finally calls the Starlark function
|
|
||||||
`reset` to reset the entire state of the plugin.
|
|
||||||
|
|
||||||
The Starlark functions can use the global function `state` to keep temporary the
|
|
||||||
metrics to aggregate.
|
|
||||||
|
|
||||||
The Starlark language is a dialect of Python, and will be familiar to those who
|
|
||||||
have experience with the Python language. However, there are major
|
have experience with the Python language. However, there are major
|
||||||
[differences](#python-differences). Existing
|
[differences](#python-differences). Existing Python code is unlikely to work
|
||||||
Python code is unlikely to work unmodified. The execution environment is
|
unmodified.
|
||||||
sandboxed, and it is not possible to do I/O operations such as reading from
|
|
||||||
files or sockets.
|
|
||||||
|
|
||||||
The **[Starlark specification][]** has details about the syntax and available
|
> [!NOTE]
|
||||||
functions.
|
> The execution environment is sandboxed, and it is not possible to access the
|
||||||
|
> local filesystem or perfoming network operations. This is by design of the
|
||||||
|
> Starlark language as a configuration language.
|
||||||
|
|
||||||
|
The Starlark script used by this plugin needs to be composed of the three
|
||||||
|
methods defining an aggreagtor named `add`, `push` and `reset`.
|
||||||
|
|
||||||
|
The `add` method is called as soon as a new metric is added to the plugin the
|
||||||
|
metrics to the aggregator. After `period`, the `push` method is called to
|
||||||
|
output the resulting metrics and finally the aggregation is reset by using the
|
||||||
|
`reset` method of the Starlark script.
|
||||||
|
|
||||||
|
The Starlark functions might use the global function `state` to keep aggregation
|
||||||
|
information such as added metrics etc.
|
||||||
|
|
||||||
|
More details on the syntax and available functions can be found in the
|
||||||
|
[Starlark specification][spec].
|
||||||
|
|
||||||
|
⭐ Telegraf v1.21.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
|
[starlark]: https://github.com/google/starlark-go
|
||||||
|
[spec]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
@ -95,7 +106,7 @@ def reset():
|
||||||
```
|
```
|
||||||
|
|
||||||
For a list of available types and functions that can be used in the code, see
|
For a list of available types and functions that can be used in the code, see
|
||||||
the [Starlark specification][].
|
the [Starlark specification][spec].
|
||||||
|
|
||||||
## Python Differences
|
## Python Differences
|
||||||
|
|
||||||
|
|
@ -117,11 +128,9 @@ documentation about the Starlark processor.
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
- [minmax](testdata/min_max.star) - A minmax aggregator implemented with a Starlark script.
|
- [minmax](testdata/min_max.star)
|
||||||
- [merge](testdata/merge.star) - A merge aggregator implemented with a Starlark script.
|
- [merge](testdata/merge.star)
|
||||||
|
|
||||||
[All examples](testdata) are in the testdata folder.
|
[All examples](testdata) are in the testdata folder.
|
||||||
|
|
||||||
Open a Pull Request to add any other useful Starlark examples.
|
Open a Pull Request to add any other useful Starlark examples.
|
||||||
|
|
||||||
[Starlark specification]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md
|
|
||||||
|
|
|
||||||
|
|
@ -1,19 +1,23 @@
|
||||||
# Value Counter Aggregator Plugin
|
# Value Counter Aggregator Plugin
|
||||||
|
|
||||||
The valuecounter plugin counts the occurrence of values in fields and emits the
|
This plugin counts the occurrence of unique values in fields and emits the
|
||||||
counter once every 'period' seconds.
|
counter once every `period` with the field-names being suffixed by the unique
|
||||||
|
value converted to `string`.
|
||||||
|
|
||||||
A use case for the valuecounter plugin is when you are processing a HTTP access
|
> [!NOTE]
|
||||||
log (with the logparser input) and want to count the HTTP status codes.
|
> The fields to be counted must be configured using the `fields` setting,
|
||||||
|
> otherwise no field will be counted and no metric is emitted.
|
||||||
|
|
||||||
The fields which will be counted must be configured with the `fields`
|
This plugin is useful to e.g. count the occurrances of HTTP status codes or
|
||||||
configuration directive. When no `fields` is provided the plugin will not count
|
other categorical values in the defined `period`.
|
||||||
any fields. The results are emitted in fields in the format:
|
|
||||||
`originalfieldname_fieldvalue = count`.
|
|
||||||
|
|
||||||
Counting fields with a high number of potential values may produce significant
|
> [!IMPORTANT]
|
||||||
amounts of new fields and memory usage, take care to only count fields with a
|
> Counting fields with a high number of potential values may produce a
|
||||||
limited set of values.
|
> significant amounts of new fields and results in an increased memory usage.
|
||||||
|
> Take care to only count fields with a limited set of values.
|
||||||
|
|
||||||
|
⭐ Telegraf v1.8.0
|
||||||
|
💻 all
|
||||||
|
|
||||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue