docs(specs): Add specification for custom-builder (#14901)

Co-authored-by: Joshua Powers <powersj@fastmail.com>
Co-authored-by: Dane Strandboge <136023093+DStrand1@users.noreply.github.com>

docs/specs/tsd-002-custom-builder.md

@@ -0,0 +1,71 @@
+# Telegraf Custom-Builder
+
+## Objective
+
+Provide a tool to build a customized, smaller version of Telegraf with only
+the required plugins included.
+
+## Keywords
+
+tool, binary size, customization
+
+## Overview
+
+The Telegraf binary continues to grow as new plugins and features are added
+and dependencies are updated. Users running on resource constrained systems
+such as embedded-systems or inside containers might suffer from the growth.
+
+This document specifies a tool to build a smaller Telegraf binary tailored to
+the plugins configured and actually used, removing unnecessary and unused
+plugins. The implementation should be able to cope with configured parsers and
+serializers including defaults for those plugin categories. Valid Telegraf
+configuration files, including directories containing such files, are the input
+to the customization process.
+
+The customization tool might not be available for older versions of Telegraf.
+Furthermore, the degree of customization and thus the effective size reduction
+might vary across versions. The tool must create a single static Telegraf
+binary. Distribution packages or containers are *not* targeted.
+
+## Prior art
+
+[PR #5809](https://github.com/influxdata/telegraf/pull/5809) and
+[telegraf-lite-builder](https://github.com/influxdata/telegraf/tree/telegraf-lite-builder/cmd/telegraf-lite-builder):
+
+- Uses docker
+- Uses browser:
+  - Generates a webpage to pick what options you want. User chooses plugins;
+    does not take a config file
+  - Build a binary, then minifies by stripping and compressing that binary
+- Does some steps that belong in makefile, not builder
+  - Special case for upx
+  - Makes gzip, zip, tar.gz
+- Uses gopkg.in?
+- Can also work from the command line
+
+[PR #8519](https://github.com/influxdata/telegraf/pull/8519)
+
+- User chooses plugins OR provides a config file
+
+[powers/telegraf-build](https://github.com/powersj/telegraf-build)
+
+- User chooses plugins OR provides a config file
+- Currently kept in separate repo
+- Undoes changes to all.go files
+
+[rawkode/bring-your-own-telegraf](https://github.com/rawkode/bring-your-own-telegraf)
+
+- Uses docker
+
+## Additional information
+
+You might be able to further reduce the binary size of Telegraf by removing
+debugging information. This is done by adding `-w` and `-s` to the linker flags
+before building `LDFLAGS="-w -s"`.
+
+However, please note that this removes information helpful for debugging issues
+in Telegraf.
+
+Additionally, you can use a binary packer such as [UPX](https://upx.github.io/)
+to reduce the required *disk* space. This compresses the binary and decompresses
+it again at runtime. However, this does not reduce memory footprint at runtime.

tools/custom_builder/README.md

@@ -4,6 +4,29 @@ Telegraf's `custom_builder` is a tool to select the plugins compiled into the
 Telegraf binary. By doing so, Telegraf can become smaller, saving both disk
 space and memory if only a sub-set of plugins is selected.
 
+## Requirements
+
+For compiling the customized binary you need the
+[Golang language](https://go.dev/) as well as the `make` build system.
+The minimum required version of Golang can be found in the *Build From Source*
+section of the `README.md` file of your version. Both the `go` and the `make`
+command must be available in your path.
+
+## Downloading code
+
+The first step is to download the Telegraf repository for the version you are
+planning to customize. In the example below, we want to use `v1.29.5` but you
+might also use other versions or `master`.
+
+```shell
+# git clone --branch v1.29.5 --single-branch https://github.com/influxdata/telegraf.git
+...
+# cd telegraf
+```
+
+Alternatively, you can download the source tarball or zip-archive of a
+[Telegraf release](https://github.com/influxdata/telegraf/releases).
+
 ## Building
 
 To build `custom_builder` run the following command:
@@ -16,18 +39,17 @@ The resulting binary is located in the `tools/custom_builder` folder.
 
 ## Running
 
-The easiest way of building a customized Telegraf is to use your
+The easiest way of building a customized Telegraf is to use your Telegraf
-Telegraf configuration file(s). Assuming your configuration is
+configuration file(s). Assuming your configuration is in
-in `/etc/telegraf/telegraf.conf` you can run
+`/etc/telegraf/telegraf.conf` you can run
 
 ```shell
 # ./tools/custom_builder/custom_builder --config /etc/telegraf/telegraf.conf
 ```
 
-to build a Telegraf binary tailored to your configuration.
+to build a Telegraf binary tailored to your configuration. You can also specify
-You can also specify a configuration directory similar to
+a configuration directory similar to Telegraf itself. To additionally use the
-Telegraf itself. To additionally use the configurations in
+configurations in `/etc/telegraf/telegraf.d` run
-`/etc/telegraf/telegraf.d` run
 
 ```shell
 # ./tools/custom_builder/custom_builder                      \
@@ -35,8 +57,8 @@ Telegraf itself. To additionally use the configurations in
     --config-dir /etc/telegraf/telegraf.d
 ```
 
-Configurations can also be retrieved from remote locations just
+Configurations can also be retrieved from remote locations just like for
-like for Telegraf.
+Telegraf.
 
 ```shell
 # ./tools/custom_builder/custom_builder --config http://myserver/telegraf.conf
@@ -44,11 +66,10 @@ like for Telegraf.
 
 will download the configuration from `myserver`.
 
-The `--config` and `--config-dir` option can be used multiple times.
+The `--config` and `--config-dir` option can be used multiple times. In case
-In case you want to deploy Telegraf to multiple systems with
+you want to deploy Telegraf to multiple systems with different configurations,
-different configurations, simply specify the super-set of all
+simply specify the super-set of all configurations you have. `custom_builder`
-configurations you have. `custom_builder` will figure out the list
+will figure out the list for you
-for you
 
 ```shell
 # ./tools/custom_builder/custom_builder             \
@@ -74,8 +95,8 @@ To get more help run
 
 ## Notes
 
-Please make sure to include all `parsers` you intend to use and check the
+Please make sure to include all `parsers` and `serializers` you intend to use
-enabled-plugins list.
+and check the enabled-plugins list.
 
-Additional plugins can potentially be enabled automatically due to
+Additional plugins can potentially be enabled automatically due to dependencies
-dependencies without being shown in the enabled-plugins list.
+without being shown in the enabled-plugins list.