From 73be345d8806537332a76451c4aae758b29a18c4 Mon Sep 17 00:00:00 2001 From: Joshua Powers Date: Thu, 16 May 2024 01:41:13 -0600 Subject: [PATCH] docs(inputs): Clean up, add note about external connections (#15356) --- docs/INPUTS.md | 208 +++++++++++++++++++++++++++---------------------- 1 file changed, 114 insertions(+), 94 deletions(-) diff --git a/docs/INPUTS.md b/docs/INPUTS.md index 0908a643d..029a1d568 100644 --- a/docs/INPUTS.md +++ b/docs/INPUTS.md @@ -20,14 +20,125 @@ and submit new inputs. - Each plugin requires a file called `sample.conf` containing the sample configuration for the plugin in TOML format. Please consult the [Sample Config][] page for the latest style guidelines. -- Each plugin `README.md` file should include the `sample.conf` file in a section - describing the configuration by specifying a `toml` section in the form `toml @sample.conf`. The specified file(s) are then injected automatically into the Readme. +- Each plugin `README.md` file should include the `sample.conf` file in a + section describing the configuration by specifying a `toml` section in the + form `toml @sample.conf`. The specified file(s) are then injected + automatically into the Readme. - Follow the recommended [Code Style][]. +[Sample Config]: /docs/developers/SAMPLE_CONFIG.md +[Code Style]: /docs/developers/CODE_STYLE.md +[telegraf.Input]: https://godoc.org/github.com/influxdata/telegraf#Input + +### Typed Metrics + +In addition to the `AddFields` function, the accumulator also supports +functions to add typed metrics: `AddGauge`, `AddCounter`, etc. Metric types +are ignored by the InfluxDB output, but can be used for other outputs, such as +[prometheus][prom metric types]. + +[prom metric types]: https://prometheus.io/docs/concepts/metric_types/ + +### Data Formats + +Some input plugins, such as the [exec][] plugin, can accept any supported +[input data formats][]. + +In order to enable this, you must specify a `SetParser(parser parsers.Parser)` +function on the plugin object (see the exec plugin for an example), as well as +defining `parser` as a field of the object. + +You can then utilize the parser internally in your plugin, parsing data as you +see fit. Telegraf's configuration layer will take care of instantiating and +creating the `Parser` object. + +Add the following to the sample configuration in the README.md: + +```toml + ## Data format to consume. + ## Each data format has its own unique set of configuration options, read + ## more about them here: + ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md + data_format = "influx" +``` + +[exec]: /plugins/inputs/exec +[input data formats]: /docs/DATA_FORMATS_INPUT.md + +### Service Input Plugins + +This section is for developers who want to create new "service" collection +inputs. A service plugin differs from a regular plugin in that it operates a +background service while Telegraf is running. One example would be the +`statsd` plugin, which operates a statsd server. + +Service Input Plugins are substantially more complicated than a regular +plugin, as they will require threads and locks to verify data integrity. +Service Input Plugins should be avoided unless there is no way to create their +behavior with a regular plugin. + +To create a Service Input implement the [telegraf.ServiceInput][] interface. + +[telegraf.ServiceInput]: https://godoc.org/github.com/influxdata/telegraf#ServiceInput + +### Metric Tracking + +Metric Tracking provides a system to be notified when metrics have been +successfully written to their outputs or otherwise discarded. This allows +inputs to be created that function as reliable queue consumers. + +To get started with metric tracking begin by calling `WithTracking` on the +[telegraf.Accumulator][]. Add metrics using the `AddTrackingMetricGroup` +function on the returned [telegraf.TrackingAccumulator][] and store the +`TrackingID`. The `Delivered()` channel will return a type with information +about the final delivery status of the metric group. + +Check the [amqp_consumer][] for an example implementation. + +[telegraf.Accumulator]: https://godoc.org/github.com/influxdata/telegraf#Accumulator +[telegraf.TrackingAccumulator]: https://godoc.org/github.com/influxdata/telegraf#Accumulator +[amqp_consumer]: /plugins/inputs/amqp_consumer + +### External Services + +Plugins that connect or require the use of external services should ensure that +those servers are active. When may depend on the type of input plugin: + +For service input plugins, `Init` should be used to check for configuration +issues (e.g. bad option) and for other non-recoverable errors. Then `Start` +is used to create connections or other retry-able operations. + +For normal inputs, `Init` should also be used to check for configuration issues +as well as any other dependencies that the plugin will require. For example, +any binaries that must exist for the plugin to function. If making a connection, +this should also take place in `Init`. + +Developers may find that they switch to using service input plugins more and +more to take advantage of the error on retry behavior features. This allows +the user to decide what to do on an error, like ignoring the plugin or retrying +constantly. + +## Input Plugin Example + Let's say you've written a plugin that emits metrics about processes on the current host. -## Input Plugin Example +### Register Plugin + +Registration of the plugin on `plugins/inputs/all/simple.go`: + +```go +//go:build !custom || inputs || inputs.simple + +package all + +import _ "github.com/influxdata/telegraf/plugins/inputs/simple" // register plugin +``` + +The _build-tags_ in the first line allow to selectively include/exclude your +plugin when customizing Telegraf. + +### Plugin Content of your plugin file e.g. `simple.go` @@ -73,94 +184,3 @@ func init() { inputs.Add("simple", func() telegraf.Input { return &Simple{} }) } ``` - -Registration of the plugin on `plugins/inputs/all/simple.go`: - -```go -//go:build !custom || inputs || inputs.simple - -package all - -import _ "github.com/influxdata/telegraf/plugins/inputs/simple" // register plugin - -``` - -The _build-tags_ in the first line allow to selectively include/exclude your -plugin when customizing Telegraf. - -### Development - -- Run `make static` followed by `make plugin-[pluginName]` to spin up a docker - dev environment using docker-compose. -- __[Optional]__ When developing a plugin, add a `dev` directory with a - `docker-compose.yml` and `telegraf.conf` as well as any other supporting - files, where sensible. - -### Typed Metrics - -In addition to the `AddFields` function, the accumulator also supports -functions to add typed metrics: `AddGauge`, `AddCounter`, etc. Metric types -are ignored by the InfluxDB output, but can be used for other outputs, such as -[prometheus][prom metric types]. - -### Data Formats - -Some input plugins, such as the [exec][] plugin, can accept any supported -[input data formats][]. - -In order to enable this, you must specify a `SetParser(parser parsers.Parser)` -function on the plugin object (see the exec plugin for an example), as well as -defining `parser` as a field of the object. - -You can then utilize the parser internally in your plugin, parsing data as you -see fit. Telegraf's configuration layer will take care of instantiating and -creating the `Parser` object. - -Add the following to the sample configuration in the README.md: - -```toml - ## Data format to consume. - ## Each data format has its own unique set of configuration options, read - ## more about them here: - ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md - data_format = "influx" -``` - -### Service Input Plugins - -This section is for developers who want to create new "service" collection -inputs. A service plugin differs from a regular plugin in that it operates a -background service while Telegraf is running. One example would be the -`statsd` plugin, which operates a statsd server. - -Service Input Plugins are substantially more complicated than a regular -plugin, as they will require threads and locks to verify data integrity. -Service Input Plugins should be avoided unless there is no way to create their -behavior with a regular plugin. - -To create a Service Input implement the [telegraf.ServiceInput][] interface. - -### Metric Tracking - -Metric Tracking provides a system to be notified when metrics have been -successfully written to their outputs or otherwise discarded. This allows -inputs to be created that function as reliable queue consumers. - -To get started with metric tracking begin by calling `WithTracking` on the -[telegraf.Accumulator][]. Add metrics using the `AddTrackingMetricGroup` -function on the returned [telegraf.TrackingAccumulator][] and store the -`TrackingID`. The `Delivered()` channel will return a type with information -about the final delivery status of the metric group. - -Check the [amqp_consumer][] for an example implementation. - -[exec]: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec -[amqp_consumer]: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/amqp_consumer -[prom metric types]: https://prometheus.io/docs/concepts/metric_types/ -[input data formats]: https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md -[Sample Config]: https://github.com/influxdata/telegraf/blob/master/docs/developers/SAMPLE_CONFIG.md -[Code Style]: https://github.com/influxdata/telegraf/blob/master/docs/developers/CODE_STYLE.md -[telegraf.Input]: https://godoc.org/github.com/influxdata/telegraf#Input -[telegraf.ServiceInput]: https://godoc.org/github.com/influxdata/telegraf#ServiceInput -[telegraf.Accumulator]: https://godoc.org/github.com/influxdata/telegraf#Accumulator -[telegraf.TrackingAccumulator]: https://godoc.org/github.com/influxdata/telegraf#Accumulator