2020-08-12 04:10:41 +08:00
|
|
|
# Kafka Consumer Legacy Input Plugin
|
2017-06-08 09:22:28 +08:00
|
|
|
|
2022-06-08 05:37:08 +08:00
|
|
|
**Deprecated in version 1.4. Please use [Kafka Consumer input plugin][]**
|
2021-11-17 06:11:24 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
The [Kafka](http://kafka.apache.org/) consumer plugin polls a specified Kafka
|
2022-06-08 05:37:08 +08:00
|
|
|
topic and adds messages to InfluxDB. The plugin assumes messages follow the line
|
|
|
|
|
protocol. [Consumer Group][1] is used to talk to the Kafka cluster so multiple
|
|
|
|
|
instances of telegraf can read from the same topic in parallel.
|
|
|
|
|
|
|
|
|
|
[1]: http://godoc.org/github.com/wvanbergen/kafka/consumergroup
|
2017-06-08 09:22:28 +08:00
|
|
|
|
2023-03-27 20:12:35 +08:00
|
|
|
## Service Input <!-- @/docs/includes/service_input.md -->
|
|
|
|
|
|
|
|
|
|
This plugin is a service input. Normal plugins gather metrics determined by the
|
|
|
|
|
interval setting. Service plugins start a service to listens and waits for
|
|
|
|
|
metrics or events to occur. Service plugins have two key differences from
|
|
|
|
|
normal plugins:
|
|
|
|
|
|
|
|
|
|
1. The global or plugin specific `interval` setting may not apply
|
|
|
|
|
2. The CLI options of `--test`, `--test-wait`, and `--once` may not produce
|
|
|
|
|
output for this plugin
|
|
|
|
|
|
2022-10-27 03:58:36 +08:00
|
|
|
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
|
|
|
|
|
|
|
|
|
|
In addition to the plugin-specific configuration settings, plugins support
|
|
|
|
|
additional global and plugin configuration settings. These settings are used to
|
|
|
|
|
modify metrics, tags, and field or create aliases and configure ordering, etc.
|
|
|
|
|
See the [CONFIGURATION.md][CONFIGURATION.md] for more details.
|
|
|
|
|
|
2023-01-12 23:55:21 +08:00
|
|
|
[CONFIGURATION.md]: ../../../docs/CONFIGURATION.md#plugins
|
2022-10-27 03:58:36 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
## Configuration
|
|
|
|
|
|
2022-05-24 21:49:47 +08:00
|
|
|
```toml @sample.conf
|
2017-06-08 09:22:28 +08:00
|
|
|
# Read metrics from Kafka topic(s)
|
2022-04-08 06:01:21 +08:00
|
|
|
[[inputs.kafka_consumer_legacy]]
|
2017-06-08 09:22:28 +08:00
|
|
|
## topic(s) to consume
|
|
|
|
|
topics = ["telegraf"]
|
2019-09-24 06:39:50 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
## an array of Zookeeper connection strings
|
|
|
|
|
zookeeper_peers = ["localhost:2181"]
|
2019-09-24 06:39:50 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
## Zookeeper Chroot
|
|
|
|
|
zookeeper_chroot = ""
|
2019-09-24 06:39:50 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
## the name of the consumer group
|
|
|
|
|
consumer_group = "telegraf_metrics_consumers"
|
2019-09-24 06:39:50 +08:00
|
|
|
|
2017-06-08 09:22:28 +08:00
|
|
|
## Offset (must be either "oldest" or "newest")
|
|
|
|
|
offset = "oldest"
|
|
|
|
|
|
|
|
|
|
## Data format to consume.
|
|
|
|
|
## Each data format has its own unique set of configuration options, read
|
|
|
|
|
## more about them here:
|
|
|
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
|
|
|
|
data_format = "influx"
|
|
|
|
|
|
|
|
|
|
## Maximum length of a message to consume, in bytes (default 0/unlimited);
|
|
|
|
|
## larger messages are dropped
|
|
|
|
|
max_message_len = 65536
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Testing
|
|
|
|
|
|
|
|
|
|
Running integration tests requires running Zookeeper & Kafka. See Makefile
|
|
|
|
|
for kafka container command.
|
2021-11-25 03:59:41 +08:00
|
|
|
|
2022-06-08 05:37:08 +08:00
|
|
|
[Kafka Consumer input plugin]: ../kafka_consumer/README.md
|
2023-02-09 18:04:41 +08:00
|
|
|
|
|
|
|
|
## Metrics
|
|
|
|
|
|
|
|
|
|
## Example Output
|