docs(specs): Add specification for custom-builder (#14901)
Co-authored-by: Joshua Powers <powersj@fastmail.com> Co-authored-by: Dane Strandboge <136023093+DStrand1@users.noreply.github.com>
This commit is contained in:
Sven Rebhan
GitHub
parent
1ab29b1190
commit
73f80f4adf
|
|
@ -0,0 +1,71 @@
|
|||
# Telegraf Custom-Builder
|
||||
|
||||
## Objective
|
||||
|
||||
Provide a tool to build a customized, smaller version of Telegraf with only
|
||||
the required plugins included.
|
||||
|
||||
## Keywords
|
||||
|
||||
tool, binary size, customization
|
||||
|
||||
## Overview
|
||||
|
||||
The Telegraf binary continues to grow as new plugins and features are added
|
||||
and dependencies are updated. Users running on resource constrained systems
|
||||
such as embedded-systems or inside containers might suffer from the growth.
|
||||
|
||||
This document specifies a tool to build a smaller Telegraf binary tailored to
|
||||
the plugins configured and actually used, removing unnecessary and unused
|
||||
plugins. The implementation should be able to cope with configured parsers and
|
||||
serializers including defaults for those plugin categories. Valid Telegraf
|
||||
configuration files, including directories containing such files, are the input
|
||||
to the customization process.
|
||||
|
||||
The customization tool might not be available for older versions of Telegraf.
|
||||
Furthermore, the degree of customization and thus the effective size reduction
|
||||
might vary across versions. The tool must create a single static Telegraf
|
||||
binary. Distribution packages or containers are *not* targeted.
|
||||
|
||||
## Prior art
|
||||
|
||||
[PR #5809](https://github.com/influxdata/telegraf/pull/5809) and
|
||||
[telegraf-lite-builder](https://github.com/influxdata/telegraf/tree/telegraf-lite-builder/cmd/telegraf-lite-builder):
|
||||
|
||||
- Uses docker
|
||||
- Uses browser:
|
||||
- Generates a webpage to pick what options you want. User chooses plugins;
|
||||
does not take a config file
|
||||
- Build a binary, then minifies by stripping and compressing that binary
|
||||
- Does some steps that belong in makefile, not builder
|
||||
- Special case for upx
|
||||
- Makes gzip, zip, tar.gz
|
||||
- Uses gopkg.in?
|
||||
- Can also work from the command line
|
||||
|
||||
[PR #8519](https://github.com/influxdata/telegraf/pull/8519)
|
||||
|
||||
- User chooses plugins OR provides a config file
|
||||
|
||||
[powers/telegraf-build](https://github.com/powersj/telegraf-build)
|
||||
|
||||
- User chooses plugins OR provides a config file
|
||||
- Currently kept in separate repo
|
||||
- Undoes changes to all.go files
|
||||
|
||||
[rawkode/bring-your-own-telegraf](https://github.com/rawkode/bring-your-own-telegraf)
|
||||
|
||||
- Uses docker
|
||||
|
||||
## Additional information
|
||||
|
||||
You might be able to further reduce the binary size of Telegraf by removing
|
||||
debugging information. This is done by adding `-w` and `-s` to the linker flags
|
||||
before building `LDFLAGS="-w -s"`.
|
||||
|
||||
However, please note that this removes information helpful for debugging issues
|
||||
in Telegraf.
|
||||
|
||||
Additionally, you can use a binary packer such as [UPX](https://upx.github.io/)
|
||||
to reduce the required *disk* space. This compresses the binary and decompresses
|
||||
it again at runtime. However, this does not reduce memory footprint at runtime.
|
||||
|
|
@ -4,6 +4,29 @@ Telegraf's `custom_builder` is a tool to select the plugins compiled into the
|
|||
Telegraf binary. By doing so, Telegraf can become smaller, saving both disk
|
||||
space and memory if only a sub-set of plugins is selected.
|
||||
|
||||
## Requirements
|
||||
|
||||
For compiling the customized binary you need the
|
||||
[Golang language](https://go.dev/) as well as the `make` build system.
|
||||
The minimum required version of Golang can be found in the *Build From Source*
|
||||
section of the `README.md` file of your version. Both the `go` and the `make`
|
||||
command must be available in your path.
|
||||
|
||||
## Downloading code
|
||||
|
||||
The first step is to download the Telegraf repository for the version you are
|
||||
planning to customize. In the example below, we want to use `v1.29.5` but you
|
||||
might also use other versions or `master`.
|
||||
|
||||
```shell
|
||||
# git clone --branch v1.29.5 --single-branch https://github.com/influxdata/telegraf.git
|
||||
...
|
||||
# cd telegraf
|
||||
```
|
||||
|
||||
Alternatively, you can download the source tarball or zip-archive of a
|
||||
[Telegraf release](https://github.com/influxdata/telegraf/releases).
|
||||
|
||||
## Building
|
||||
|
||||
To build `custom_builder` run the following command:
|
||||
|
|
@ -16,18 +39,17 @@ The resulting binary is located in the `tools/custom_builder` folder.
|
|||
|
||||
## Running
|
||||
|
||||
The easiest way of building a customized Telegraf is to use your
|
||||
Telegraf configuration file(s). Assuming your configuration is
|
||||
in `/etc/telegraf/telegraf.conf` you can run
|
||||
The easiest way of building a customized Telegraf is to use your Telegraf
|
||||
configuration file(s). Assuming your configuration is in
|
||||
`/etc/telegraf/telegraf.conf` you can run
|
||||
|
||||
```shell
|
||||
# ./tools/custom_builder/custom_builder --config /etc/telegraf/telegraf.conf
|
||||
```
|
||||
|
||||
to build a Telegraf binary tailored to your configuration.
|
||||
You can also specify a configuration directory similar to
|
||||
Telegraf itself. To additionally use the configurations in
|
||||
`/etc/telegraf/telegraf.d` run
|
||||
to build a Telegraf binary tailored to your configuration. You can also specify
|
||||
a configuration directory similar to Telegraf itself. To additionally use the
|
||||
configurations in `/etc/telegraf/telegraf.d` run
|
||||
|
||||
```shell
|
||||
# ./tools/custom_builder/custom_builder \
|
||||
|
|
@ -35,8 +57,8 @@ Telegraf itself. To additionally use the configurations in
|
|||
--config-dir /etc/telegraf/telegraf.d
|
||||
```
|
||||
|
||||
Configurations can also be retrieved from remote locations just
|
||||
like for Telegraf.
|
||||
Configurations can also be retrieved from remote locations just like for
|
||||
Telegraf.
|
||||
|
||||
```shell
|
||||
# ./tools/custom_builder/custom_builder --config http://myserver/telegraf.conf
|
||||
|
|
@ -44,11 +66,10 @@ like for Telegraf.
|
|||
|
||||
will download the configuration from `myserver`.
|
||||
|
||||
The `--config` and `--config-dir` option can be used multiple times.
|
||||
In case you want to deploy Telegraf to multiple systems with
|
||||
different configurations, simply specify the super-set of all
|
||||
configurations you have. `custom_builder` will figure out the list
|
||||
for you
|
||||
The `--config` and `--config-dir` option can be used multiple times. In case
|
||||
you want to deploy Telegraf to multiple systems with different configurations,
|
||||
simply specify the super-set of all configurations you have. `custom_builder`
|
||||
will figure out the list for you
|
||||
|
||||
```shell
|
||||
# ./tools/custom_builder/custom_builder \
|
||||
|
|
@ -74,8 +95,8 @@ To get more help run
|
|||
|
||||
## Notes
|
||||
|
||||
Please make sure to include all `parsers` you intend to use and check the
|
||||
enabled-plugins list.
|
||||
Please make sure to include all `parsers` and `serializers` you intend to use
|
||||
and check the enabled-plugins list.
|
||||
|
||||
Additional plugins can potentially be enabled automatically due to
|
||||
dependencies without being shown in the enabled-plugins list.
|
||||
Additional plugins can potentially be enabled automatically due to dependencies
|
||||
without being shown in the enabled-plugins list.
|
||||
|
|
|
|||
Loading…
Reference in New Issue