910b726876
The previous implementation of SeriesGrouper required breaking a metric object apart into its constituents, converting tags and keys into unoptimized maps, only to have it put them back together into another metric object. This resulted in a significant performance overhead. This overhead was further compounded when the number of fields was large.
This change adds a new AddMetric method to SeriesGrouper which preserves the metric object and removes the back-and-forth conversion.
Additionlly the method used for calculating the metric's hash was switched to use maphash, which is optimized for this case.
----
Benchmarks
Before:
BenchmarkMergeOne-16 106012 11790 ns/op
BenchmarkMergeTwo-16 48529 24819 ns/op
BenchmarkGroupID-16 780018 1608 ns/op
After:
BenchmarkMergeOne-16 907093 1173 ns/op
BenchmarkMergeTwo-16 508321 2168 ns/op
BenchmarkGroupID-16 11217788 99.4 ns/op
|
||
|---|---|---|
| .. | ||
| README.md | ||
| merge.go | ||
| merge_test.go | ||
README.md
Merge Aggregator
Merge metrics together into a metric with multiple fields into the most memory and network transfer efficient form.
Use this plugin when fields are split over multiple metrics, with the same measurement, tag set and timestamp. By merging into a single metric they can be handled more efficiently by the output.
Configuration
[[aggregators.merge]]
## If true, the original metric will be dropped by the
## aggregator and will not get sent to the output plugins.
drop_original = true
Example
- cpu,host=localhost usage_time=42 1567562620000000000
- cpu,host=localhost idle_time=42 1567562620000000000
+ cpu,host=localhost idle_time=42,usage_time=42 1567562620000000000