docs(aggregators): Add plugin metadata and update description (#16089)
This commit is contained in:
Sven Rebhan
GitHub
parent
6045e13616
commit
d76723d21c
|
|
@ -1,8 +1,11 @@
|
|||
# BasicStats Aggregator Plugin
|
||||
# Basic Statistics Aggregator Plugin
|
||||
|
||||
The BasicStats aggregator plugin gives count, diff, max, min, mean,
|
||||
non_negative_diff, sum, s2(variance), stdev for a set of values, last and
|
||||
first, emitting the aggregate every `period` seconds.
|
||||
This plugin computes basic statistics such as counts, differences, minima,
|
||||
maxima, mean values, non-negative differences etc. for a set of metrics and
|
||||
emits these statistical values every `period`.
|
||||
|
||||
⭐ Telegraf v1.5.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
# Derivative Aggregator Plugin
|
||||
|
||||
The Derivative Aggregator Plugin estimates the derivative for all fields of the
|
||||
aggregated metrics.
|
||||
This plugin computes the derivative for all fields of the aggregated metrics.
|
||||
|
||||
⭐ Telegraf v1.18.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -1,15 +1,24 @@
|
|||
# Final Aggregator Plugin
|
||||
|
||||
The final aggregator emits the last metric of a contiguous series. A
|
||||
contiguous series is defined as a series which receives updates within the
|
||||
time period in `series_timeout`. The contiguous series may be longer than the
|
||||
time interval defined by `period`.
|
||||
This plugin emits the last metric of a contiguous series, defined as a
|
||||
series which receives updates within the time period in `series_timeout`. The
|
||||
contiguous series may be longer than the time interval defined by `period`.
|
||||
When a series has not been updated within the `series_timeout`, the last metric
|
||||
is emitted.
|
||||
|
||||
Alternatively, the plugin emits the last metric in the `period` for the
|
||||
`periodic` output strategy.
|
||||
|
||||
This is useful for getting the final value for data sources that produce
|
||||
discrete time series such as procstat, cgroup, kubernetes etc.
|
||||
discrete time series such as procstat, cgroup, kubernetes etc. or to downsample
|
||||
metrics collected at a higher frequency.
|
||||
|
||||
When a series has not been updated within the time defined in
|
||||
`series_timeout`, the last metric is emitted with the `_final` appended.
|
||||
> [!NOTE]
|
||||
> All emited metrics do have fields with `_final` appended to the field-name
|
||||
> by default.
|
||||
|
||||
⭐ Telegraf v1.11.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -1,33 +1,21 @@
|
|||
# Histogram Aggregator Plugin
|
||||
|
||||
The histogram aggregator plugin creates histograms containing the counts of
|
||||
field values within a range.
|
||||
This plugin creates histograms containing the counts of field values within the
|
||||
configured range. The histogram metric is emitted every `period`.
|
||||
|
||||
If `cumulative` is set to true, values added to a bucket are also added to the
|
||||
larger buckets in the distribution. This creates a [cumulative histogram][1].
|
||||
Otherwise, values are added to only one bucket, which creates an [ordinary
|
||||
histogram][1]
|
||||
In `cumulative` mode, values added to a bucket are also added to the
|
||||
consecutive buckets in the distribution creating a [cumulative histogram][1].
|
||||
|
||||
Like other Telegraf aggregators, the metric is emitted every `period` seconds.
|
||||
By default bucket counts are not reset between periods and will be non-strictly
|
||||
increasing while Telegraf is running. This behavior can be changed by setting
|
||||
the `reset` parameter to true.
|
||||
> [!NOTE]
|
||||
> By default bucket counts are not reset between periods and will be
|
||||
> non-strictly increasing while Telegraf is running. This behavior can be
|
||||
> by setting the `reset` parameter.
|
||||
|
||||
⭐ Telegraf v1.4.0
|
||||
💻 all
|
||||
|
||||
[1]: https://en.wikipedia.org/wiki/Histogram#/media/File:Cumulative_vs_normal_histogram.svg
|
||||
|
||||
## Design
|
||||
|
||||
Each metric is passed to the aggregator and this aggregator searches histogram
|
||||
buckets for those fields, which have been specified in the config. If buckets
|
||||
are found, the aggregator will increment +1 to the appropriate
|
||||
bucket. Otherwise, it will be added to the `+Inf` bucket. Every `period`
|
||||
seconds this data will be forwarded to the outputs.
|
||||
|
||||
The algorithm of hit counting to buckets was implemented on the base of the
|
||||
algorithm which is implemented in the Prometheus [client][2].
|
||||
|
||||
[2]: https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
In addition to the plugin-specific configuration settings, plugins support
|
||||
|
|
|
|||
|
|
@ -1,11 +1,14 @@
|
|||
# Merge Aggregator Plugin
|
||||
|
||||
Merge metrics together into a metric with multiple fields into the most memory
|
||||
and network transfer efficient form.
|
||||
This plugin merges metrics of the same series and timestamp into new metrics
|
||||
with the super-set of fields. A series here is defined by the metric name and
|
||||
the tag key-value set.
|
||||
|
||||
Use this plugin when fields are split over multiple metrics, with the same
|
||||
measurement, tag set and timestamp. By merging into a single metric they can
|
||||
be handled more efficiently by the output.
|
||||
measurement, tag set and timestamp.
|
||||
|
||||
⭐ Telegraf v1.13.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,11 @@
|
|||
# MinMax Aggregator Plugin
|
||||
# Minimum-Maximum Aggregator Plugin
|
||||
|
||||
The minmax aggregator plugin aggregates min & max values of each field it sees,
|
||||
emitting the aggrate every `period` seconds.
|
||||
This plugin aggregates the minimum and maximum values of each field it sees,
|
||||
emitting the aggrate every `period` seconds with field names suffixed by `_min`
|
||||
and `_max` respectively.
|
||||
|
||||
⭐ Telegraf v1.1.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,11 @@
|
|||
# Quantile Aggregator Plugin
|
||||
|
||||
The quantile aggregator plugin aggregates specified quantiles for each numeric
|
||||
field per metric it sees and emits the quantiles every `period`.
|
||||
This plugin aggregates each numeric field per metric into the specified
|
||||
quantiles and emits the quantiles every `period`. Different aggregation
|
||||
algorithms are supported with varying accuracy and limitations.
|
||||
|
||||
⭐ Telegraf v1.18.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
@ -135,11 +139,6 @@ cpu,cpu=cpu-total,host=Hugin usage_idle=85.78199052131747,usage_nice=0,usage_irq
|
|||
cpu,cpu=cpu-total,host=Hugin usage_guest_nice_075=0,usage_user_050=10.814851731872487,usage_guest_075=0,usage_steal_025=0,usage_irq_025=1.031558489546918,usage_irq_075=1.0471206791944527,usage_iowait_025=0,usage_guest_050=0,usage_guest_nice_050=0,usage_nice_075=0,usage_iowait_050=0,usage_system_050=2.1601016518428664,usage_irq_050=1.046598554697342,usage_guest_nice_025=0,usage_idle_050=85.79616247197244,usage_softirq_075=0.1887208672481664,usage_steal_075=0,usage_system_025=2.0778058770562287,usage_system_075=2.1640279004292173,usage_softirq_050=0.1778907242693666,usage_nice_050=0,usage_iowait_075=0.01270648030495347,usage_user_075=10.895078647178611,usage_nice_025=0,usage_steal_050=0,usage_user_025=10.04529117724472,usage_idle_025=85.78907649664495,usage_idle_075=86.57025404411868,usage_softirq_025=0.1761619083594677,usage_guest_025=0 1608288390000000000
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- Dunning & Ertl: "Computing Extremely Accurate Quantiles Using t-Digests", arXiv:1902.04023 (2019) [pdf][tdigest_paper]
|
||||
- Hyndman & Fan: "Sample Quantiles in Statistical Packages", The American Statistician, vol. 50, pp. 361-365 (1996) [pdf][hyndman_fan]
|
||||
|
||||
[tdigest_paper]: https://arxiv.org/abs/1902.04023
|
||||
[tdigest_lib]: https://github.com/caio/go-tdigest
|
||||
[hyndman_fan]: http://www.maths.usyd.edu.au/u/UG/SM/STAT3022/r/current/Misc/Sample%20Quantiles%20in%20Statistical%20Packages.pdf
|
||||
|
|
|
|||
|
|
@ -1,26 +1,37 @@
|
|||
# Starlark Aggregator Plugin
|
||||
|
||||
The `starlark` aggregator allows to implement a custom aggregator plugin with a
|
||||
Starlark script. The Starlark script needs to be composed of the three methods
|
||||
defined in the Aggregator plugin interface which are `add`, `push` and `reset`.
|
||||
This plugin allows to implement a custom aggregator plugin via a
|
||||
[Starlark][starlark] script.
|
||||
|
||||
The Starlark Aggregator plugin calls the Starlark function `add` to add the
|
||||
metrics to the aggregator, then calls the Starlark function `push` to push the
|
||||
resulting metrics into the accumulator and finally calls the Starlark function
|
||||
`reset` to reset the entire state of the plugin.
|
||||
|
||||
The Starlark functions can use the global function `state` to keep temporary the
|
||||
metrics to aggregate.
|
||||
|
||||
The Starlark language is a dialect of Python, and will be familiar to those who
|
||||
The Starlark language is a dialect of Python and will be familiar to those who
|
||||
have experience with the Python language. However, there are major
|
||||
[differences](#python-differences). Existing
|
||||
Python code is unlikely to work unmodified. The execution environment is
|
||||
sandboxed, and it is not possible to do I/O operations such as reading from
|
||||
files or sockets.
|
||||
[differences](#python-differences). Existing Python code is unlikely to work
|
||||
unmodified.
|
||||
|
||||
The **[Starlark specification][]** has details about the syntax and available
|
||||
functions.
|
||||
> [!NOTE]
|
||||
> The execution environment is sandboxed, and it is not possible to access the
|
||||
> local filesystem or perfoming network operations. This is by design of the
|
||||
> Starlark language as a configuration language.
|
||||
|
||||
The Starlark script used by this plugin needs to be composed of the three
|
||||
methods defining an aggreagtor named `add`, `push` and `reset`.
|
||||
|
||||
The `add` method is called as soon as a new metric is added to the plugin the
|
||||
metrics to the aggregator. After `period`, the `push` method is called to
|
||||
output the resulting metrics and finally the aggregation is reset by using the
|
||||
`reset` method of the Starlark script.
|
||||
|
||||
The Starlark functions might use the global function `state` to keep aggregation
|
||||
information such as added metrics etc.
|
||||
|
||||
More details on the syntax and available functions can be found in the
|
||||
[Starlark specification][spec].
|
||||
|
||||
⭐ Telegraf v1.21.0
|
||||
💻 all
|
||||
|
||||
[starlark]: https://github.com/google/starlark-go
|
||||
[spec]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
@ -95,7 +106,7 @@ def reset():
|
|||
```
|
||||
|
||||
For a list of available types and functions that can be used in the code, see
|
||||
the [Starlark specification][].
|
||||
the [Starlark specification][spec].
|
||||
|
||||
## Python Differences
|
||||
|
||||
|
|
@ -117,11 +128,9 @@ documentation about the Starlark processor.
|
|||
|
||||
## Examples
|
||||
|
||||
- [minmax](testdata/min_max.star) - A minmax aggregator implemented with a Starlark script.
|
||||
- [merge](testdata/merge.star) - A merge aggregator implemented with a Starlark script.
|
||||
- [minmax](testdata/min_max.star)
|
||||
- [merge](testdata/merge.star)
|
||||
|
||||
[All examples](testdata) are in the testdata folder.
|
||||
|
||||
Open a Pull Request to add any other useful Starlark examples.
|
||||
|
||||
[Starlark specification]: https://github.com/google/starlark-go/blob/d1966c6b9fcd/doc/spec.md
|
||||
|
|
|
|||
|
|
@ -1,19 +1,23 @@
|
|||
# ValueCounter Aggregator Plugin
|
||||
# Value Counter Aggregator Plugin
|
||||
|
||||
The valuecounter plugin counts the occurrence of values in fields and emits the
|
||||
counter once every 'period' seconds.
|
||||
This plugin counts the occurrence of unique values in fields and emits the
|
||||
counter once every `period` with the field-names being suffixed by the unique
|
||||
value converted to `string`.
|
||||
|
||||
A use case for the valuecounter plugin is when you are processing a HTTP access
|
||||
log (with the logparser input) and want to count the HTTP status codes.
|
||||
> [!NOTE]
|
||||
> The fields to be counted must be configured using the `fields` setting,
|
||||
> otherwise no field will be counted and no metric is emitted.
|
||||
|
||||
The fields which will be counted must be configured with the `fields`
|
||||
configuration directive. When no `fields` is provided the plugin will not count
|
||||
any fields. The results are emitted in fields in the format:
|
||||
`originalfieldname_fieldvalue = count`.
|
||||
This plugin is useful to e.g. count the occurrances of HTTP status codes or
|
||||
other categorical values in the defined `period`.
|
||||
|
||||
Counting fields with a high number of potential values may produce significant
|
||||
amounts of new fields and memory usage, take care to only count fields with a
|
||||
limited set of values.
|
||||
> [!IMPORTANT]
|
||||
> Counting fields with a high number of potential values may produce a
|
||||
> significant amounts of new fields and results in an increased memory usage.
|
||||
> Take care to only count fields with a limited set of values.
|
||||
|
||||
⭐ Telegraf v1.8.0
|
||||
💻 all
|
||||
|
||||
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
||||
|
||||
|
|
|
|||
|
|
@ -101,7 +101,7 @@ to use them.
|
|||
# timeout="10s"
|
||||
|
||||
## MaxIdleConns controls the maximum number of idle (keep-alive) connections
|
||||
## across all hosts. Zero means unlimited.
|
||||
##across all hosts. Zero means unlimited.
|
||||
# max_idle_conn = 0
|
||||
|
||||
## MaxIdleConnsPerHost, if non-zero, controls the maximum idle (keep-alive)
|
||||
|
|
|
|||
Loading…
Reference in New Issue