CL-Softwares
/
telegraf
8
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(secretstores): Add secret-store to access OAuth2 services (#13621)

This commit is contained in:
Sven Rebhan 2023-07-14 17:26:06 +02:00 committed by GitHub
parent 6377f69501
commit 56075a9604
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 706 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
plugins/secretstores/all/oauth2.go Normal file
View File

@ -0,0 +1,5 @@
//go:build !custom || secretstores || secretstores.oauth2
package all
import _ "github.com/influxdata/telegraf/plugins/secretstores/oauth2" // register plugin

136
plugins/secretstores/oauth2/README.md Normal file
View File

@ -0,0 +1,136 @@
# OAuth2 Secret-store Plugin
The `oauth2` plugin allows to retrieve and maintain secrets from various OAuth2
services such as [Auth0][auth0], [AzureAD][azuread] or others (see
[Configuration section](#configuration)).
Tokens that are expired or are about to expire will be automatically renewed
by this secret-store, so other plugins referencing those tokens can then use
them to perform their API calls without hassle.
**Please note:** This plugin only supports the *2-legged client credentials*
flow.
You can use Telegraf to test token retrieval. Run
```shell
telegraf secrets help
```
to get more information on how to do access secrets with Telegraf.
## Usage <!-- @/docs/includes/secret_usage.md -->
Secrets defined by a store are referenced with `@{<store-id>:<secret_key>}`
the Telegraf configuration. Only certain Telegraf plugins and options of
support secret stores. To see which plugins and options support
secrets, see their respective documentation (e.g.
`plugins/outputs/influxdb/README.md`). If the plugin's README has the
`Secret-store support` section, it will detail which options support secret
store usage.
## Configuration
```toml @sample.conf
# Secret-store to retrieve and maintain tokens from various OAuth2 services
[[secretstores.oauth2]]
## Unique identifier for the secret-store.
## This id can later be used in plugins to reference the secrets
## in this secret-store via @{<id>:<secret_key>} (mandatory)
id = "secretstore"
## Service to retrieve the token(s) from
## Currently supported services are "custom", "auth0" and "AzureAD"
# service = "custom"
## Setting to overwrite the queried token-endpoint
## This setting is optional for some serices but mandatory for others such
## as "custom" or "auth0". Please check the documentation at
## https://github.com/influxdata/telegraf/blob/master/plugins/secretstores/oauth2/README.md
# token_endpoint = ""
## Tenant ID for the AzureAD service
# tenant_id = ""
## Minimal remaining time until the token expires
## If a token expires less than the set duration in the future, the token is
## renewed. This is useful to avoid race-condition issues where a token is
## still valid, but isn't when the request reaches the API endpoint of
## your service using the token.
# token_expiry_margin = "1s"
## Section for defining a token secret
[[secretstores.oauth2.token]]
## Unique secret-key used for referencing the token via @{<id>:<secret_key>}
key = ""
## Client-ID and secret for the 2-legged OAuth flow
client_id = ""
client_secret = ""
## Scopes to send in the request
# scopes = []
## Additional (optional) parameters to include in the token request
## This might for example include the "audience" parameter required for
## auth0.
# [secretstores.oauth2.token.parameters]
# audience = ""
```
All services allow multiple `[[secretstores.oauth2.token]]` sections to be
specified to define different tokens for the secret store. Please make sure to
specify `key`s that are **unique** within the secret-store instance as those
are used to reference the tokens/secrets later.
The `oauth2` secret-store supports various services that might differ in the
required or allowed settings as listed below. All of the services accept
optional `scopes` and optional `parameter` settings if not stated otherwise.
Please **replace the placeholders** in the minumal example configurations below
and add `scopes` and/or `parameters` if required.
### Auth0
To use the [Auth0 service][auth0] for retrieving the token you need to set the
`token_endpoint` to your application's endpoint. Furthermore, specifying the
`audience` parameter is required. An example configuration look like
```toml
[[secretstores.oauth2]]
id = "secretstore"
service = "auth0"
token_endpoint = "https://YOUR_DOMAIN/oauth/token"
[[secretstores.oauth2.token]]
key = "mytoken"
client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
[secretstores.oauth2.token.parameters]
audience = "YOUR_API_IDENTIFIER"
```
### AzureAD
To use the [AzureAD service][azuread] for retrieving the token you need to set
the `tenant_id` and provide a valid `scope`. An example configuration look like
```toml
[[secretstores.oauth2]]
id = "secretstore"
service = "AzureAD"
tenant_id = "YOUR_TENANT_ID"
[[secretstores.oauth2.token]]
key = "mytoken"
client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
scopes = ["YOUR_CLIENT_ID/.default"]
```
### Custom service
If your service is not listed above, you can still use it setting
`service = "custom"` as well as the `token_endpoint`. Please make sure your
service is configured for the *2-legged client credentials* OAuth2 flow!
[auth0]: https://auth0.com
[azuread]: https://azure.microsoft.com/en/products/active-directory

197
plugins/secretstores/oauth2/oauth2.go Normal file
View File

@ -0,0 +1,197 @@
//go:generate ../../../tools/readme_config_includer/generator
package oauth2
import (
"context"
_ "embed"
"errors"
"fmt"
"strings"
"time"
"golang.org/x/oauth2"
"golang.org/x/oauth2/clientcredentials"
"golang.org/x/oauth2/endpoints"
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/config"
"github.com/influxdata/telegraf/plugins/secretstores"
)
//go:embed sample.conf
var sampleConfig string
type TokenConfig struct {
Key string `toml:"key"`
ClientID config.Secret `toml:"client_id"`
ClientSecret config.Secret `toml:"client_secret"`
Scopes []string `toml:"scopes"`
Params map[string]string `toml:"parameters"`
}
type OAuth2 struct {
Service string `toml:"service"`
Endpoint string `toml:"token_endpoint"`
Tenant string `toml:"tenant_id"`
ExpiryMargin config.Duration `toml:"token_expiry_margin"`
TokenConfigs []TokenConfig `toml:"token"`
Log telegraf.Logger `toml:"-"`
sources map[string]oauth2.TokenSource
cancel context.CancelFunc
}
func (*OAuth2) SampleConfig() string {
return sampleConfig
}
// Init initializes all internals of the secret-store
func (o *OAuth2) Init() error {
ctx, cancel := context.WithCancel(context.Background())
o.cancel = cancel
// Check the service setting and determine the endpoint
var endpoint oauth2.Endpoint
var requireTenant, acceptCustomEndpoint bool
switch strings.ToLower(o.Service) {
case "", "custom":
if o.Endpoint == "" {
return errors.New("'token_endpoint' required for custom service")
}
endpoint.TokenURL = o.Endpoint
endpoint.AuthStyle = oauth2.AuthStyleAutoDetect
acceptCustomEndpoint = true
case "auth0":
if o.Endpoint == "" {
return errors.New("'token_endpoint' required for Auth0")
}
endpoint = oauth2.Endpoint{
TokenURL: o.Endpoint,
AuthStyle: oauth2.AuthStyleInParams,
}
acceptCustomEndpoint = true
case "azuread":
if o.Tenant == "" {
return errors.New("'tenant_id' required for AzureAD")
}
requireTenant = true
endpoint = endpoints.AzureAD(o.Tenant)
default:
return fmt.Errorf("service %q not supported", o.Service)
}
if !requireTenant && o.Tenant != "" {
o.Log.Warnf("'tenant_id' set but ignored by service %q", o.Service)
}
if !acceptCustomEndpoint && o.Endpoint != "" {
return fmt.Errorf("'token_endpoint' cannot be set for service %q", o.Service)
}
// Setup the token sources
o.sources = make(map[string]oauth2.TokenSource, len(o.TokenConfigs))
for _, c := range o.TokenConfigs {
if c.Key == "" {
return errors.New("'key' not specified")
}
if c.ClientID.Empty() {
return fmt.Errorf("'client_id' not specified for key %q", c.Key)
}
if c.ClientSecret.Empty() {
return fmt.Errorf("'client_secret' not specified for key %q", c.Key)
}
// Check service specific parameters
if strings.ToLower(o.Service) == "auth0" {
if audience := c.Params["audience"]; audience == "" {
return fmt.Errorf("'audience' parameter in key %q missing for service Auth0", c.Key)
}
}
if _, found := o.sources[c.Key]; found {
return fmt.Errorf("token with key %q already defined", c.Key)
}
// Get the secrets
cid, err := c.ClientID.Get()
if err != nil {
return fmt.Errorf("getting client ID for %q failed: %w", c.Key, err)
}
csecret, err := c.ClientSecret.Get()
if err != nil {
return fmt.Errorf("getting client secret for %q failed: %w", c.Key, err)
}
// Setup the configuration
cfg := &clientcredentials.Config{
ClientID: string(cid),
ClientSecret: string(csecret),
TokenURL: endpoint.TokenURL,
Scopes: c.Scopes,
AuthStyle: endpoint.AuthStyle,
}
config.ReleaseSecret(cid)
config.ReleaseSecret(csecret)
// Add the parameters if any
for k, v := range c.Params {
cfg.EndpointParams.Add(k, v)
}
src := cfg.TokenSource(ctx)
o.sources[c.Key] = oauth2.ReuseTokenSourceWithExpiry(nil, src, time.Duration(o.ExpiryMargin))
}
return nil
}
// Get searches for the given key and return the secret
func (o *OAuth2) Get(key string) ([]byte, error) {
src, found := o.sources[key]
if !found {
return nil, fmt.Errorf("token %q not found", key)
}
// Return the token from the token-source. The token will be automatically
// renewed if the token expires.
token, err := src.Token()
if err != nil {
return nil, err
}
if !token.Valid() {
return nil, errors.New("token invalid")
}
return []byte(token.AccessToken), nil
}
// Set sets the given secret for the given key
func (o *OAuth2) Set(_, _ string) error {
return errors.New("not supported")
}
// List lists all known secret keys
func (o *OAuth2) List() ([]string, error) {
keys := make([]string, 0, len(o.sources))
for k := range o.sources {
keys = append(keys, k)
}
return keys, nil
}
// GetResolver returns a function to resolve the given key.
func (o *OAuth2) GetResolver(key string) (telegraf.ResolveFunc, error) {
resolver := func() ([]byte, bool, error) {
s, err := o.Get(key)
return s, true, err
}
return resolver, nil
}
// Register the secret-store on load.
func init() {
secretstores.Add("oauth2", func(_ string) telegraf.SecretStore {
return &OAuth2{ExpiryMargin: config.Duration(time.Second)}
})
}

326
plugins/secretstores/oauth2/oauth2_test.go Normal file
View File

@ -0,0 +1,326 @@
package oauth2
import (
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/influxdata/telegraf/config"
"github.com/stretchr/testify/require"
)
func TestSampleConfig(t *testing.T) {
plugin := &OAuth2{}
require.NotEmpty(t, plugin.SampleConfig())
}
func TestInitFail(t *testing.T) {
tests := []struct {
name string
plugin *OAuth2
expected string
}{
{
name: "no service",
plugin: &OAuth2{},
expected: "'token_endpoint' required for custom service",
},
{
name: "custom service no URL",
plugin: &OAuth2{},
expected: "'token_endpoint' required for custom service",
},
{
name: "invalid service",
plugin: &OAuth2{Service: "foo"},
expected: `service "foo" not supported`,
},
{
name: "AzureAD without tenant",
plugin: &OAuth2{Service: "AzureAD"},
expected: "'tenant_id' required for AzureAD",
},
{
name: "token without key",
plugin: &OAuth2{
Service: "custom",
Endpoint: "http://localhost:8080",
TokenConfigs: []TokenConfig{{}}},
expected: "'key' not specified",
},
{
name: "token without client ID",
plugin: &OAuth2{
Service: "custom",
Endpoint: "http://localhost:8080",
TokenConfigs: []TokenConfig{
{
Key: "test",
},
},
},
expected: "'client_id' not specified",
},
{
name: "token without client secret",
plugin: &OAuth2{
Service: "custom",
Endpoint: "http://localhost:8080",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
},
},
},
expected: "'client_secret' not specified",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.plugin.Init()
require.ErrorContains(t, err, tt.expected)
})
}
}
func TestSetUnsupported(t *testing.T) {
plugin := &OAuth2{
Service: "custom",
Endpoint: "http://localhost:8080",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
require.ErrorContains(t, plugin.Set("foo", "bar"), "not supported")
}
func TestGetNonExisting(t *testing.T) {
plugin := &OAuth2{
Service: "custom",
Endpoint: "http://localhost:8080",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Make sure the key does not exist and try to read that key
_, err := plugin.Get("foo")
require.EqualError(t, err, `token "foo" not found`)
}
func TestResolver404(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotFound)
}))
defer server.Close()
plugin := &OAuth2{
Service: "custom",
Endpoint: server.URL + "/token",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Get the resolver
resolver, err := plugin.GetResolver("test")
require.NoError(t, err)
require.NotNil(t, resolver)
_, _, err = resolver()
require.ErrorContains(t, err, "404 Not Found")
}
func TestGet(t *testing.T) {
expected := "MTQ0NjJkZmQ5OTM2NDE1ZTZjNGZmZjI3"
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
return
}
creds := "client_id=someone&client_secret=s3cr3t&grant_type=client_credentials"
if !strings.Contains(string(body), creds) {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"access_token":"%s","scope":"read write","token_type":"bearer","expires_in":299}`, expected)
}))
defer server.Close()
plugin := &OAuth2{
Service: "custom",
Endpoint: server.URL + "/token",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Get the resolver
token, err := plugin.Get("test")
require.NoError(t, err)
require.Equal(t, expected, string(token))
}
func TestGetMultipleTimes(t *testing.T) {
expected := []string{"MTQ0NjJkZmQ5OTM2NDE1ZTZjNGZmZjI3", "03807CB390319329BDF6C777D4DFAE9C0D3B3C35"}
index := 0
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
return
}
creds := "client_id=someone&client_secret=s3cr3t&grant_type=client_credentials"
if !strings.Contains(string(body), creds) {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"access_token":"%s","scope":"read write","token_type":"bearer","expires_in":60}`, expected[index])
index++
}))
defer server.Close()
plugin := &OAuth2{
Service: "custom",
Endpoint: server.URL + "/token",
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Get the secret
token, err := plugin.Get("test")
require.NoError(t, err)
require.Equal(t, expected[0], string(token))
// Get the token another time and it should still be the same as it didn't
// expire yet.
token, err = plugin.Get("test")
require.NoError(t, err)
require.Equal(t, expected[0], string(token))
}
func TestGetExpired(t *testing.T) {
expected := "MTQ0NjJkZmQ5OTM2NDE1ZTZjNGZmZjI3"
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
return
}
creds := "client_id=someone&client_secret=s3cr3t&grant_type=client_credentials"
if !strings.Contains(string(body), creds) {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"access_token":"%s","scope":"read write","token_type":"bearer","expires_in":3}`, expected)
}))
defer server.Close()
plugin := &OAuth2{
Service: "custom",
Endpoint: server.URL + "/token",
ExpiryMargin: config.Duration(5 * time.Second),
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Get the secret
token, err := plugin.Get("test")
require.ErrorContains(t, err, "token invalid")
require.Nil(t, token)
}
func TestGetRefresh(t *testing.T) {
expected := []string{"MTQ0NjJkZmQ5OTM2NDE1ZTZjNGZmZjI3", "03807CB390319329BDF6C777D4DFAE9C0D3B3C35"}
index := 0
server := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
if err != nil {
_, _ = w.Write([]byte(err.Error()))
w.WriteHeader(http.StatusInternalServerError)
return
}
creds := "client_id=someone&client_secret=s3cr3t&grant_type=client_credentials"
if !strings.Contains(string(body), creds) {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"access_token":"%s","scope":"read write","token_type":"bearer","expires_in":6}`, expected[index])
index++
}))
defer server.Close()
plugin := &OAuth2{
Service: "custom",
Endpoint: server.URL + "/token",
ExpiryMargin: config.Duration(5 * time.Second),
TokenConfigs: []TokenConfig{
{
Key: "test",
ClientID: config.NewSecret([]byte("someone")),
ClientSecret: config.NewSecret([]byte("s3cr3t")),
},
},
}
require.NoError(t, plugin.Init())
// Get the secret
token, err := plugin.Get("test")
require.NoError(t, err)
require.Equal(t, expected[0], string(token))
// Wait until the secret expired and get the secret again
time.Sleep(2 * time.Second)
token, err = plugin.Get("test")
require.NoError(t, err)
require.Equal(t, expected[1], string(token))
}

42
plugins/secretstores/oauth2/sample.conf Normal file
View File

@ -0,0 +1,42 @@
# Secret-store to retrieve and maintain tokens from various OAuth2 services
[[secretstores.oauth2]]
## Unique identifier for the secret-store.
## This id can later be used in plugins to reference the secrets
## in this secret-store via @{<id>:<secret_key>} (mandatory)
id = "secretstore"
## Service to retrieve the token(s) from
## Currently supported services are "custom", "auth0" and "AzureAD"
# service = "custom"
## Setting to overwrite the queried token-endpoint
## This setting is optional for some serices but mandatory for others such
## as "custom" or "auth0". Please check the documentation at
## https://github.com/influxdata/telegraf/blob/master/plugins/secretstores/oauth2/README.md
# token_endpoint = ""
## Tenant ID for the AzureAD service
# tenant_id = ""
## Minimal remaining time until the token expires
## If a token expires less than the set duration in the future, the token is
## renewed. This is useful to avoid race-condition issues where a token is
## still valid, but isn't when the request reaches the API endpoint of
## your service using the token.
# token_expiry_margin = "1s"
## Section for defining a token secret
[[secretstores.oauth2.token]]
## Unique secret-key used for referencing the token via @{<id>:<secret_key>}
key = ""
## Client-ID and secret for the 2-legged OAuth flow
client_id = ""
client_secret = ""
## Scopes to send in the request
# scopes = []
## Additional (optional) parameters to include in the token request
## This might for example include the "audience" parameter required for
## auth0.
# [secretstores.oauth2.token.parameters]
# audience = ""