CL-Softwares
/
telegraf
8
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Nfsclient input: (Reopen of PR 1491 and 1684) (#4615)

This commit is contained in:
Pmoranga 2021-03-05 11:56:28 -03:00 committed by GitHub
parent 9e45322661
commit 74d4836c25
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 1088 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@ -259,6 +259,7 @@ For documentation on the latest development code see the [documentation index][d
* [net](./plugins/inputs/net)
* [net_response](./plugins/inputs/net_response)
* [netstat](./plugins/inputs/net)
* [nfsclient](./plugins/inputs/nfsclient)
* [nginx](./plugins/inputs/nginx)
* [nginx_plus_api](./plugins/inputs/nginx_plus_api)
* [nginx_plus](./plugins/inputs/nginx_plus)

1
plugins/inputs/all/all.go
View File

@ -112,6 +112,7 @@ import (
_ "github.com/influxdata/telegraf/plugins/inputs/neptune_apex"
_ "github.com/influxdata/telegraf/plugins/inputs/net"
_ "github.com/influxdata/telegraf/plugins/inputs/net_response"
_ "github.com/influxdata/telegraf/plugins/inputs/nfsclient"
_ "github.com/influxdata/telegraf/plugins/inputs/nginx"
_ "github.com/influxdata/telegraf/plugins/inputs/nginx_plus"
_ "github.com/influxdata/telegraf/plugins/inputs/nginx_plus_api"

181
plugins/inputs/nfsclient/README.md Normal file
View File

@ -0,0 +1,181 @@
#### Description
The NFSClient plugin collects data from /proc/self/mountstats. By default, only a limited number of general system-level metrics are collected, including basic read/write counts.
If `fullstat` is set, a great deal of additional metrics are collected, detailed below.
**NOTE** Many of the metrics, even if tagged with a mount point, are really _per-server_. Thus, if you mount these two shares: `nfs01:/vol/foo/bar` and `nfs01:/vol/foo/baz`, there will be two near identical entries in /proc/self/mountstats. This is a limitation of the metrics exposed by the kernel, not the telegraf plugin.
#### Plugin arguments:
- **fullstat** bool: Collect per-operation type metrics. Defaults to false.
- **include_mounts** list(string): gather metrics for only these mounts. Default is to watch all mounts.
- **exclude_mounts** list(string): gather metrics for all mounts, except those listed in this option. Excludes take precedence over includes.
- **include_operations** list(string): List of specific NFS operations to track. See /proc/self/mountstats (the "per-op statistics" section) for complete lists of valid options for NFSv3 and NFSV4. The default is to gather all metrics, but this is almost certainly *not* what you want (there are 22 operations for NFSv3, and well over 50 for NFSv4). A suggested 'minimal' list of operations to collect for basic usage: `['READ','WRITE','ACCESS','GETATTR','READDIR','LOOKUP','LOOKUP']`
- **exclude_operations** list(string): Gather all metrics, except those listed. Excludes take precedence over includes.
*N.B.* the `include_mounts` and `exclude_mounts` arguments are both applied to the local mount location (e.g. /mnt/NFS), not the server export (e.g. nfsserver:/vol/NFS). Go regexp patterns can be used in either.
#### Examples
```toml
[[inputs.nfsclient]]
## Read more low-level metrics (optional, defaults to false)
# fullstat = false
## List of mounts to explictly include or exclude (optional)
## The pattern (Go regexp) is matched against the mount point (not the
## device being mounted). If include_mounts is set, all mounts are ignored
## unless present in the list. If a mount is listed in both include_mounts
## and exclude_mounts, it is excluded. Go regexp patterns can be used.
# include_mounts = []
# exclude_mounts = []
## List of operations to include or exclude from collecting. This applies
## only when fullstat=true. Symantics are similar to {include,exclude}_mounts:
## the default is to collect everything; when include_operations is set, only
## those OPs are collected; when exclude_operations is set, all are collected
## except those listed. If include and exclude are set, the OP is excluded.
## See /proc/self/mountstats for a list of valid operations; note that
## NFSv3 and NFSv4 have different lists. While it is not possible to
## have different include/exclude lists for NFSv3/4, unused elements
## in the list should be okay. It is possible to have different lists
## for different mountpoints: use mulitple [[input.nfsclient]] stanzas,
## with their own lists. See "include_mounts" above, and be careful of
## duplicate metrics.
# include_operations = []
# exclude_operations = []
```
Example output for basic metrics showing server-wise read and write data:
```
nfsstat,mountpoint=/NFS,operation=READ,serverexport=1.2.3.4:/storage/NFS ops=600i,retrans=1i,bytes=1207i,rtt=606i,exe=607i 1612651512000000000
nfsstat,mountpoint=/NFS,operation=WRITE,serverexport=1.2.3.4:/storage/NFS bytes=1407i,rtt=706i,exe=707i,ops=700i,retrans=1i 1612651512000000000
```
Example output for `fullstat=true` metrics, which includes additional measurements for `nfs_bytes`, `nfs_events`, and `nfs_xprt_tcp` (and `nfs_xprt_udp` if present).
Additionally, per-OP metrics are collected, with examples for READ, LOOKUP, and NULL shown.
Please refer to `/proc/self/mountstats` for a list of supported NFS operations, as it changes as it changes periodically.
```
nfs_bytes,mountpoint=/home,serverexport=nfs01:/vol/home directreadbytes=0i,directwritebytes=0i,normalreadbytes=42648757667i,normalwritebytes=0i,readpages=10404603i,serverreadbytes=42617098139i,serverwritebytes=0i,writepages=0i 1608787697000000000
nfs_events,mountpoint=/home,serverexport=nfs01:/vol/home attrinvalidates=116i,congestionwait=0i,datainvalidates=65i,delay=0i,dentryrevalidates=5911243i,extendwrite=0i,inoderevalidates=200378i,pnfsreads=0i,pnfswrites=0i,setattrtrunc=0i,shortreads=0i,shortwrites=0i,sillyrenames=0i,vfsaccess=7203852i,vfsflush=117405i,vfsfsync=0i,vfsgetdents=3368i,vfslock=0i,vfslookup=740i,vfsopen=157281i,vfsreadpage=16i,vfsreadpages=86874i,vfsrelease=155526i,vfssetattr=0i,vfsupdatepage=0i,vfswritepage=0i,vfswritepages=215514i 1608787697000000000
nfs_xprt_tcp,mountpoint=/home,serverexport=nfs01:/vol/home backlogutil=0i,badxids=0i,bind_count=1i,connect_count=1i,connect_time=0i,idle_time=0i,inflightsends=15659826i,rpcreceives=2173896i,rpcsends=2173896i 1608787697000000000
nfs_ops,mountpoint=/NFS,operation=NULL,serverexport=1.2.3.4:/storage/NFS trans=0i,timeouts=0i,bytes_sent=0i,bytes_recv=0i,queue_time=0i,response_time=0i,total_time=0i,ops=0i 1612651512000000000
nfs_ops,mountpoint=/NFS,operation=READ,serverexport=1.2.3.4:/storage/NFS bytes=1207i,timeouts=602i,total_time=607i,exe=607i,trans=601i,bytes_sent=603i,bytes_recv=604i,queue_time=605i,ops=600i,retrans=1i,rtt=606i,response_time=606i 1612651512000000000
nfs_ops,mountpoint=/NFS,operation=WRITE,serverexport=1.2.3.4:/storage/NFS ops=700i,bytes=1407i,exe=707i,trans=701i,timeouts=702i,response_time=706i,total_time=707i,retrans=1i,rtt=706i,bytes_sent=703i,bytes_recv=704i,queue_time=705i 1612651512000000000
```
#### References
1. [nfsiostat](http://git.linux-nfs.org/?p=steved/nfs-utils.git;a=summary)
2. [net/sunrpc/stats.c - Linux source code](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/sunrpc/stats.c)
3. [What is in /proc/self/mountstats for NFS mounts: an introduction](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsIndex)
4. [The xprt: data for NFS mounts in /proc/self/mountstats](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsXprt)
#### Measurements & Fields
Always collected:
- nfsstat
- bytes (integer, bytes) - The total number of bytes exchanged doing this operation. This is bytes sent *and* received, including overhead *and* payload. (bytes = OP_bytes_sent + OP_bytes_recv. See nfs_ops below)
- ops (integer, count) - The number of operations of this type executed.
- retrans (integer, count) - The number of times an operation had to be retried (retrans = OP_trans - OP_ops. See nfs_ops below)
- exe (integer, miliseconds) - The number of miliseconds it took to process the operations.
- rtt (integer, miliseconds) - The round-trip time for operations.
In addition enabling `fullstat` will make many more metrics available.
#### Tags
- All measurements have the following tags:
- mountpoint - The local mountpoint, for instance: "/var/www"
- serverexport - The full server export, for instance: "nfsserver.example.org:/export"
- Measurements nfsstat and nfs_ops will also include:
- operation - the NFS operation in question. `READ` or `WRITE` for nfsstat, but potentially one of ~20 or ~50, depending on NFS version. A complete list of operations supported is visible in `/proc/self/mountstats`.
### Additional metrics
When `fullstat` is true, additional measurements are collected. Tags are the same as above.
#### NFS Operations
Most descriptions come from Reference [[3](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsIndex)] and `nfs_iostat.h`. Field order and names are the same as in `/proc/self/mountstats` and the Kernel source.
Please refer to `/proc/self/mountstats` for a list of supported NFS operations, as it changes occasionally.
- nfs_bytes
- fields:
- normalreadbytes - (int, bytes) - Bytes read from the server via `read()`
- normalwritebytes - (int, bytes) - Bytes written to the server via `write()`
- directreadbytes - (int, bytes) - Bytes read with O_DIRECT set
- directwritebytes - (int, bytes) -Bytes written with O_DIRECT set
- serverreadbytes - (int, bytes) - Bytes read via NFS READ (via `mmap()`)
- serverwritebytes - (int, bytes) - Bytes written via NFS WRITE (via `mmap()`)
- readpages - (int, count) - Number of pages read
- writepages - (int, count) - Number of pages written
- nfs_events - Per-event metrics
- fields:
- inoderevalidates - (int, count) - How many times cached inode attributes have to be re-validated from the server.
- dentryrevalidates - (int, count) - How many times cached dentry nodes have to be re-validated.
- datainvalidates - (int, count) - How many times an inode had its cached data thrown out.
- attrinvalidates - (int, count) - How many times an inode has had cached inode attributes invalidated.
- vfsopen - (int, count) - How many times files or directories have been `open()`'d.
- vfslookup - (int, count) - How many name lookups in directories there have been.
- vfsaccess - (int, count) - Number of calls to `access()`. (formerly called "vfspermission")
- vfsupdatepage - (int, count) - Count of updates (and potential writes) to pages.
- vfsreadpage - (int, count) - Number of pages read.
- vfsreadpages - (int, count) - Count of how many times a _group_ of pages was read (possibly via `mmap()`?).
- vfswritepage - (int, count) - Number of pages written.
- vfswritepages - (int, count) - Count of how many times a _group_ of pages was written (possibly via `mmap()`?)
- vfsgetdents - (int, count) - Count of directory entry reads with getdents(). These reads can be served from cache and don't necessarily imply actual NFS requests. (formerly called "vfsreaddir")
- vfssetattr - (int, count) - How many times we've set attributes on inodes.
- vfsflush - (int, count) - Count of times pending writes have been forcibly flushed to the server.
- vfsfsync - (int, count) - Count of calls to `fsync()` on directories and files.
- vfslock - (int, count) - Number of times a lock was attempted on a file (regardless of success or not).
- vfsrelease - (int, count) - Number of calls to `close()`.
- congestionwait - (int, count) - Believe unused by the Linux kernel, but it is part of the NFS spec.
- setattrtrunc - (int, count) - How many times files have had their size truncated.
- extendwrite - (int, count) - How many times a file has been grown because you're writing beyond the existing end of the file.
- sillyrenames - (int, count) - Number of times an in-use file was removed (thus creating a temporary ".nfsXXXXXX" file)
- shortreads - (int, count) - Number of times the NFS server returned less data than requested.
- shortwrites - (int, count) - Number of times NFS server reports it wrote less data than requested.
- delay - (int, count) - Occurances of EJUKEBOX ("Jukebox Delay", probably unused)
- pnfsreads - (int, count) - Count of NFS v4.1+ pNFS reads.
- pnfswrites - (int, count) - Count of NFS v4.1+ pNFS writes.
- nfs_xprt_tcp
- fields:
- bind_count - (int, count) - Number of _completely new_ mounts to this server (sometimes 0?)
- connect_count - (int, count) - How many times the client has connected to the server in question
- connect_time - (int, jiffies) - How long the NFS client has spent waiting for its connection(s) to the server to be established.
- idle_time - (int, seconds) - How long (in seconds) since the NFS mount saw any RPC traffic.
- rpcsends - (int, count) - How many RPC requests this mount has sent to the server.
- rpcreceives - (int, count) - How many RPC replies this mount has received from the server.
- badxids - (int, count) - Count of XIDs sent by the server that the client doesn't know about.
- inflightsends - (int, count) - Number of outstanding requests; always >1. (See reference #4 for comment on this field)
- backlogutil - (int, count) - Cumulative backlog count
- nfs_xprt_udp
- fields:
- [same as nfs_xprt_tcp, except for connect_count, connect_time, and idle_time]
- nfs_ops
- fields (In all cases, the `operations` tag is set to the uppercase name of the NFS operation, _e.g._ "READ", "FSINFO", _etc_. See /proc/self/mountstats for a full list):
- ops - (int, count) - Total operations of this type.
- trans - (int, count) - Total transmissions of this type, including retransmissions: `OP_ops - OP_trans = total_retransmissions` (lower is better).
- timeouts - (int, count) - Number of major timeouts.
- bytes_sent - (int, count) - Bytes received, including headers (should also be close to on-wire size).
- bytes_recv - (int, count) - Bytes sent, including headers (should be close to on-wire size).
- queue_time - (int, milliseconds) - Cumulative time a request waited in the queue before sending this OP type.
- response_time - (int, milliseconds) - Cumulative time waiting for a response for this OP type.
- total_time - (int, milliseconds) - Cumulative time a request waited in the queue before sending.
- errors - (int, count) - Total number operations that complete with tk_status < 0 (usually errors). This is a new field, present in kernel >=5.3, mountstats version 1.1

497
plugins/inputs/nfsclient/nfsclient.go Normal file
View File

@ -0,0 +1,497 @@
package nfsclient
import (
"bufio"
"log"
"os"
"regexp"
"strconv"
"strings"
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/internal/choice"
"github.com/influxdata/telegraf/plugins/inputs"
)
type NFSClient struct {
Fullstat bool `toml:"fullstat"`
IncludeMounts []string `toml:"include_mounts"`
ExcludeMounts []string `toml:"exclude_mounts"`
IncludeOperations []string `toml:"include_operations"`
ExcludeOperations []string `toml:"exclude_operations"`
Log telegraf.Logger `toml:"-"`
nfs3Ops map[string]bool
nfs4Ops map[string]bool
mountstatsPath string
}
const sampleConfig = `
## Read more low-level metrics (optional, defaults to false)
# fullstat = false
## List of mounts to explictly include or exclude (optional)
## The pattern (Go regexp) is matched against the mount point (not the
## device being mounted). If include_mounts is set, all mounts are ignored
## unless present in the list. If a mount is listed in both include_mounts
## and exclude_mounts, it is excluded. Go regexp patterns can be used.
# include_mounts = []
# exclude_mounts = []
## List of operations to include or exclude from collecting. This applies
## only when fullstat=true. Symantics are similar to {include,exclude}_mounts:
## the default is to collect everything; when include_operations is set, only
## those OPs are collected; when exclude_operations is set, all are collected
## except those listed. If include and exclude are set, the OP is excluded.
## See /proc/self/mountstats for a list of valid operations; note that
## NFSv3 and NFSv4 have different lists. While it is not possible to
## have different include/exclude lists for NFSv3/4, unused elements
## in the list should be okay. It is possible to have different lists
## for different mountpoints: use mulitple [[input.nfsclient]] stanzas,
## with their own lists. See "include_mounts" above, and be careful of
## duplicate metrics.
# include_operations = []
# exclude_operations = []
`
func (n *NFSClient) SampleConfig() string {
return sampleConfig
}
func (n *NFSClient) Description() string {
return "Read per-mount NFS client metrics from /proc/self/mountstats"
}
func convertToInt64(line []string) []int64 {
/* A "line" of input data (a pre-split array of strings) is
processed one field at a time. Each field is converted to
an int64 value, and appened to an array of return values.
On an error, check for ErrRange, and throw a fatal error
if found. This situation indicates a pretty major issue in
the /proc/self/mountstats file, and returning faulty data
is worse than no data. Other errors are ignored, and append
whatever we got in the first place (probably 0).
Yes, this is ugly. */
var nline []int64
if len(line) < 2 {
return nline
}
// Skip the first field; it's handled specially as the "first" variable
for _, l := range line[1:] {
val, err := strconv.ParseInt(l, 10, 64)
if err != nil {
if numError, ok := err.(*strconv.NumError); ok {
if numError.Err == strconv.ErrRange {
log.Fatalf("ErrRange: line:[%v] raw:[%v] -> parsed:[%v]\n", line, l, val)
}
}
}
nline = append(nline, val)
}
return nline
}
func (n *NFSClient) parseStat(mountpoint string, export string, version string, line []string, fullstat bool, acc telegraf.Accumulator) error {
tags := map[string]string{"mountpoint": mountpoint, "serverexport": export}
nline := convertToInt64(line)
if len(nline) == 0 {
n.Log.Warnf("Parsing Stat line with one field: %s\n", line)
return nil
}
first := strings.Replace(line[0], ":", "", 1)
var eventsFields = []string{
"inoderevalidates",
"dentryrevalidates",
"datainvalidates",
"attrinvalidates",
"vfsopen",
"vfslookup",
"vfsaccess",
"vfsupdatepage",
"vfsreadpage",
"vfsreadpages",
"vfswritepage",
"vfswritepages",
"vfsgetdents",
"vfssetattr",
"vfsflush",
"vfsfsync",
"vfslock",
"vfsrelease",
"congestionwait",
"setattrtrunc",
"extendwrite",
"sillyrenames",
"shortreads",
"shortwrites",
"delay",
"pnfsreads",
"pnfswrites",
}
var bytesFields = []string{
"normalreadbytes",
"normalwritebytes",
"directreadbytes",
"directwritebytes",
"serverreadbytes",
"serverwritebytes",
"readpages",
"writepages",
}
var xprtudpFields = []string{
"bind_count",
"rpcsends",
"rpcreceives",
"badxids",
"inflightsends",
"backlogutil",
}
var xprttcpFields = []string{
"bind_count",
"connect_count",
"connect_time",
"idle_time",
"rpcsends",
"rpcreceives",
"badxids",
"inflightsends",
"backlogutil",
}
var nfsopFields = []string{
"ops",
"trans",
"timeouts",
"bytes_sent",
"bytes_recv",
"queue_time",
"response_time",
"total_time",
"errors",
}
var fields = make(map[string]interface{})
switch first {
case "READ", "WRITE":
fields["ops"] = nline[0]
fields["retrans"] = nline[1] - nline[0]
fields["bytes"] = nline[3] + nline[4]
fields["rtt"] = nline[6]
fields["exe"] = nline[7]
tags["operation"] = first
acc.AddFields("nfsstat", fields, tags)
}
if fullstat {
switch first {
case "events":
if len(nline) >= len(eventsFields) {
for i, t := range eventsFields {
fields[t] = nline[i]
}
acc.AddFields("nfs_events", fields, tags)
}
case "bytes":
if len(nline) >= len(bytesFields) {
for i, t := range bytesFields {
fields[t] = nline[i]
}
acc.AddFields("nfs_bytes", fields, tags)
}
case "xprt":
if len(line) > 1 {
switch line[1] {
case "tcp":
if len(nline)+2 >= len(xprttcpFields) {
for i, t := range xprttcpFields {
fields[t] = nline[i+2]
}
acc.AddFields("nfs_xprt_tcp", fields, tags)
}
case "udp":
if len(nline)+2 >= len(xprtudpFields) {
for i, t := range xprtudpFields {
fields[t] = nline[i+2]
}
acc.AddFields("nfs_xprt_udp", fields, tags)
}
}
}
}
if (version == "3" && n.nfs3Ops[first]) || (version == "4" && n.nfs4Ops[first]) {
tags["operation"] = first
if len(nline) <= len(nfsopFields) {
for i, t := range nline {
fields[nfsopFields[i]] = t
}
acc.AddFields("nfs_ops", fields, tags)
}
}
}
return nil
}
func (n *NFSClient) processText(scanner *bufio.Scanner, acc telegraf.Accumulator) error {
var mount string
var version string
var export string
var skip bool
for scanner.Scan() {
line := strings.Fields(scanner.Text())
line_len := len(line)
if line_len == 0 {
continue
}
skip = false
// This denotes a new mount has been found, so set
// mount and export, and stop skipping (for now)
if line_len > 4 && choice.Contains("fstype", line) && (choice.Contains("nfs", line) || choice.Contains("nfs4", line)) {
mount = line[4]
export = line[1]
} else if line_len > 5 && (choice.Contains("(nfs)", line) || choice.Contains("(nfs4)", line)) {
version = strings.Split(line[5], "/")[1]
}
if mount == "" {
continue
}
if len(n.IncludeMounts) > 0 {
skip = true
for _, RE := range n.IncludeMounts {
matched, _ := regexp.MatchString(RE, mount)
if matched {
skip = false
break
}
}
}
if !skip && len(n.ExcludeMounts) > 0 {
for _, RE := range n.ExcludeMounts {
matched, _ := regexp.MatchString(RE, mount)
if matched {
skip = true
break
}
}
}
if !skip {
n.parseStat(mount, export, version, line, n.Fullstat, acc)
}
}
return nil
}
func (n *NFSClient) getMountStatsPath() string {
path := "/proc/self/mountstats"
if os.Getenv("MOUNT_PROC") != "" {
path = os.Getenv("MOUNT_PROC")
}
n.Log.Debugf("using [%s] for mountstats", path)
return path
}
func (n *NFSClient) Gather(acc telegraf.Accumulator) error {
file, err := os.Open(n.mountstatsPath)
if err != nil {
n.Log.Errorf("Failed opening the [%s] file: %s ", file, err)
return err
}
defer file.Close()
scanner := bufio.NewScanner(file)
n.processText(scanner, acc)
if err := scanner.Err(); err != nil {
n.Log.Errorf("%s", err)
return err
}
return nil
}
func (n *NFSClient) Init() error {
var nfs3Fields = []string{
"NULL",
"GETATTR",
"SETATTR",
"LOOKUP",
"ACCESS",
"READLINK",
"READ",
"WRITE",
"CREATE",
"MKDIR",
"SYMLINK",
"MKNOD",
"REMOVE",
"RMDIR",
"RENAME",
"LINK",
"READDIR",
"READDIRPLUS",
"FSSTAT",
"FSINFO",
"PATHCONF",
"COMMIT",
}
var nfs4Fields = []string{
"NULL",
"READ",
"WRITE",
"COMMIT",
"OPEN",
"OPEN_CONFIRM",
"OPEN_NOATTR",
"OPEN_DOWNGRADE",
"CLOSE",
"SETATTR",
"FSINFO",
"RENEW",
"SETCLIENTID",
"SETCLIENTID_CONFIRM",
"LOCK",
"LOCKT",
"LOCKU",
"ACCESS",
"GETATTR",
"LOOKUP",
"LOOKUP_ROOT",
"REMOVE",
"RENAME",
"LINK",
"SYMLINK",
"CREATE",
"PATHCONF",
"STATFS",
"READLINK",
"READDIR",
"SERVER_CAPS",
"DELEGRETURN",
"GETACL",
"SETACL",
"FS_LOCATIONS",
"RELEASE_LOCKOWNER",
"SECINFO",
"FSID_PRESENT",
"EXCHANGE_ID",
"CREATE_SESSION",
"DESTROY_SESSION",
"SEQUENCE",
"GET_LEASE_TIME",
"RECLAIM_COMPLETE",
"LAYOUTGET",
"GETDEVICEINFO",
"LAYOUTCOMMIT",
"LAYOUTRETURN",
"SECINFO_NO_NAME",
"TEST_STATEID",
"FREE_STATEID",
"GETDEVICELIST",
"BIND_CONN_TO_SESSION",
"DESTROY_CLIENTID",
"SEEK",
"ALLOCATE",
"DEALLOCATE",
"LAYOUTSTATS",
"CLONE",
"COPY",
"OFFLOAD_CANCEL",
"LOOKUPP",
"LAYOUTERROR",
"COPY_NOTIFY",
"GETXATTR",
"SETXATTR",
"LISTXATTRS",
"REMOVEXATTR",
}
nfs3Ops := make(map[string]bool)
nfs4Ops := make(map[string]bool)
n.mountstatsPath = n.getMountStatsPath()
if len(n.IncludeOperations) == 0 {
for _, Op := range nfs3Fields {
nfs3Ops[Op] = true
}
for _, Op := range nfs4Fields {
nfs4Ops[Op] = true
}
} else {
for _, Op := range n.IncludeOperations {
nfs3Ops[Op] = true
}
for _, Op := range n.IncludeOperations {
nfs4Ops[Op] = true
}
}
if len(n.ExcludeOperations) > 0 {
for _, Op := range n.ExcludeOperations {
if nfs3Ops[Op] {
delete(nfs3Ops, Op)
}
if nfs4Ops[Op] {
delete(nfs4Ops, Op)
}
}
}
if len(n.IncludeMounts) > 0 {
n.Log.Debugf("Including these mount patterns: %v", n.IncludeMounts)
} else {
n.Log.Debugf("Including all mounts.")
}
if len(n.ExcludeMounts) > 0 {
n.Log.Debugf("Excluding these mount patterns: %v", n.ExcludeMounts)
} else {
n.Log.Debugf("Not excluding any mounts.")
}
if len(n.IncludeOperations) > 0 {
n.Log.Debugf("Including these operations: %v", n.IncludeOperations)
} else {
n.Log.Debugf("Including all operations.")
}
if len(n.ExcludeOperations) > 0 {
n.Log.Debugf("Excluding these mount patterns: %v", n.ExcludeOperations)
} else {
n.Log.Debugf("Not excluding any operations.")
}
return nil
}
func init() {
inputs.Add("nfsclient", func() telegraf.Input {
return &NFSClient{}
})
}

177
plugins/inputs/nfsclient/nfsclient_test.go Normal file
View File

@ -0,0 +1,177 @@
package nfsclient
import (
"bufio"
"github.com/influxdata/telegraf/testutil"
"os"
"strings"
"testing"
)
func getMountStatsPath() string {
path := "./testdata/mountstats"
if os.Getenv("MOUNT_PROC") != "" {
path = os.Getenv("MOUNT_PROC")
}
return path
}
func TestNFSClientParsev3(t *testing.T) {
var acc testutil.Accumulator
nfsclient := NFSClient{}
nfsclient.nfs3Ops = map[string]bool{"READLINK": true, "GETATTR": false}
nfsclient.nfs4Ops = map[string]bool{"READLINK": true, "GETATTR": false}
data := strings.Fields(" READLINK: 500 501 502 503 504 505 506 507")
nfsclient.parseStat("1.2.3.4:/storage/NFS", "/A", "3", data, true, &acc)
fields_ops := map[string]interface{}{
"ops": int64(500),
"trans": int64(501),
"timeouts": int64(502),
"bytes_sent": int64(503),
"bytes_recv": int64(504),
"queue_time": int64(505),
"response_time": int64(506),
"total_time": int64(507),
}
acc.AssertContainsFields(t, "nfs_ops", fields_ops)
}
func TestNFSClientParsev4(t *testing.T) {
var acc testutil.Accumulator
nfsclient := NFSClient{}
nfsclient.nfs3Ops = map[string]bool{"DESTROY_SESSION": true, "GETATTR": false}
nfsclient.nfs4Ops = map[string]bool{"DESTROY_SESSION": true, "GETATTR": false}
data := strings.Fields(" DESTROY_SESSION: 500 501 502 503 504 505 506 507")
nfsclient.parseStat("2.2.2.2:/nfsdata/", "/B", "4", data, true, &acc)
fields_ops := map[string]interface{}{
"ops": int64(500),
"trans": int64(501),
"timeouts": int64(502),
"bytes_sent": int64(503),
"bytes_recv": int64(504),
"queue_time": int64(505),
"response_time": int64(506),
"total_time": int64(507),
}
acc.AssertContainsFields(t, "nfs_ops", fields_ops)
}
func TestNFSClientProcessStat(t *testing.T) {
var acc testutil.Accumulator
nfsclient := NFSClient{}
nfsclient.Fullstat = false
file, _ := os.Open(getMountStatsPath())
defer file.Close()
scanner := bufio.NewScanner(file)
nfsclient.processText(scanner, &acc)
fields_readstat := map[string]interface{}{
"ops": int64(600),
"retrans": int64(1),
"bytes": int64(1207),
"rtt": int64(606),
"exe": int64(607),
}
read_tags := map[string]string{
"serverexport": "1.2.3.4:/storage/NFS",
"mountpoint": "/A",
"operation": "READ",
}
acc.AssertContainsTaggedFields(t, "nfsstat", fields_readstat, read_tags)
fields_writestat := map[string]interface{}{
"ops": int64(700),
"retrans": int64(1),
"bytes": int64(1407),
"rtt": int64(706),
"exe": int64(707),
}
write_tags := map[string]string{
"serverexport": "1.2.3.4:/storage/NFS",
"mountpoint": "/A",
"operation": "WRITE",
}
acc.AssertContainsTaggedFields(t, "nfsstat", fields_writestat, write_tags)
}
func TestNFSClientProcessFull(t *testing.T) {
var acc testutil.Accumulator
nfsclient := NFSClient{}
nfsclient.Fullstat = true
file, _ := os.Open(getMountStatsPath())
defer file.Close()
scanner := bufio.NewScanner(file)
nfsclient.processText(scanner, &acc)
fields_events := map[string]interface{}{
"inoderevalidates": int64(301736),
"dentryrevalidates": int64(22838),
"datainvalidates": int64(410979),
"attrinvalidates": int64(26188427),
"vfsopen": int64(27525),
"vfslookup": int64(9140),
"vfsaccess": int64(114420),
"vfsupdatepage": int64(30785253),
"vfsreadpage": int64(5308856),
"vfsreadpages": int64(5364858),
"vfswritepage": int64(30784819),
"vfswritepages": int64(79832668),
"vfsgetdents": int64(170),
"vfssetattr": int64(64),
"vfsflush": int64(18194),
"vfsfsync": int64(29294718),
"vfslock": int64(0),
"vfsrelease": int64(18279),
"congestionwait": int64(0),
"setattrtrunc": int64(2),
"extendwrite": int64(785551),
"sillyrenames": int64(0),
"shortreads": int64(0),
"shortwrites": int64(0),
"delay": int64(0),
"pnfsreads": int64(0),
"pnfswrites": int64(0),
}
fields_bytes := map[string]interface{}{
"normalreadbytes": int64(204440464584),
"normalwritebytes": int64(110857586443),
"directreadbytes": int64(783170354688),
"directwritebytes": int64(296174954496),
"serverreadbytes": int64(1134399088816),
"serverwritebytes": int64(407107155723),
"readpages": int64(85749323),
"writepages": int64(30784819),
}
fields_xprt_tcp := map[string]interface{}{
"bind_count": int64(1),
"connect_count": int64(1),
"connect_time": int64(0),
"idle_time": int64(0),
"rpcsends": int64(96172963),
"rpcreceives": int64(96172963),
"badxids": int64(0),
"inflightsends": int64(620878754),
"backlogutil": int64(0),
}
acc.AssertContainsFields(t, "nfs_events", fields_events)
acc.AssertContainsFields(t, "nfs_bytes", fields_bytes)
acc.AssertContainsFields(t, "nfs_xprt_tcp", fields_xprt_tcp)
}

231
plugins/inputs/nfsclient/testdata/mountstats vendored Normal file
View File

@ -0,0 +1,231 @@
device rootfs mounted on / with fstype rootfs
device proc mounted on /proc with fstype proc
device sysfs mounted on /sys with fstype sysfs
device devtmpfs mounted on /dev with fstype devtmpfs
device devpts mounted on /dev/pts with fstype devpts
device tmpfs mounted on /dev/shm with fstype tmpfs
device /dev/loop0 mounted on /dev/.initramfs/live with fstype iso9660
device /dev/loop6 mounted on / with fstype ext4
device /proc/bus/usb mounted on /proc/bus/usb with fstype usbfs
device none mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
device /tmp mounted on /tmp with fstype tmpfs
device /home mounted on /home with fstype tmpfs
device /var mounted on /var with fstype tmpfs
device /etc mounted on /etc with fstype tmpfs
device /dev/ram1 mounted on /root with fstype ext2
device cgroup mounted on /cgroup/cpuset with fstype cgroup
device cgroup mounted on /cgroup/cpu with fstype cgroup
device cgroup mounted on /cgroup/cpuacct with fstype cgroup
device cgroup mounted on /cgroup/memory with fstype cgroup
device cgroup mounted on /cgroup/devices with fstype cgroup
device cgroup mounted on /cgroup/freezer with fstype cgroup
device cgroup mounted on /cgroup/net_cls with fstype cgroup
device cgroup mounted on /cgroup/blkio with fstype cgroup
device sunrpc mounted on /var/lib/nfs/rpc_pipefs with fstype rpc_pipefs
device /etc/auto.misc mounted on /misc with fstype autofs
device -hosts mounted on /net with fstype autofs
device 1.2.3.4:/storage/NFS mounted on /A with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=60,acregmax=60,acdirmin=60,acdirmax=60,hard,nolock,noacl,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=1.2.3.4,mountvers=3,mountport=49193,mountproto=tcp,local_lock=all
age: 1136770
caps: caps=0x3fe6,wtmult=512,dtsize=8192,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 301736 22838 410979 26188427 27525 9140 114420 30785253 5308856 5364858 30784819 79832668 170 64 18194 29294718 0 18279 0 2 785551 0 0 0 0 0 0
bytes: 204440464584 110857586443 783170354688 296174954496 1134399088816 407107155723 85749323 30784819
RPC iostats version: 1.0 p/v: 100003/3 (nfs)
xprt: tcp 733 1 1 0 0 96172963 96172963 0 620878754 0 690 196347132 524706275
per-op statistics
NULL: 0 0 0 0 0 0 0 0
GETATTR: 100 101 102 103 104 105 106 107
SETATTR: 200 201 202 203 204 205 206 207
LOOKUP: 300 301 302 303 304 305 306 307
ACCESS: 400 401 402 403 404 405 406 407
READLINK: 500 501 502 503 504 505 506 507
READ: 600 601 602 603 604 605 606 607
WRITE: 700 701 702 703 704 705 706 707
CREATE: 800 801 802 803 804 805 806 807
MKDIR: 900 901 902 903 904 905 906 907
SYMLINK: 1000 1001 1002 1003 1004 1005 1006 1007
MKNOD: 1100 1101 1102 1103 1104 1105 1106 1107
REMOVE: 1200 1201 1202 1203 1204 1205 1206 1207
RMDIR: 1300 1301 1302 1303 1304 1305 1306 1307
RENAME: 1400 1401 1402 1403 1404 1405 1406 1407
LINK: 1500 1501 1502 1503 1504 1505 1506 1507
READDIR: 1600 1601 1602 1603 1604 1605 1606 1607
READDIRPLUS: 1700 1701 1702 1703 1704 1705 1706 1707
FSSTAT: 1800 1801 1802 1803 1804 1805 1806 1807
FSINFO: 1900 1901 1902 1903 1904 1905 1906 1907
PATHCONF: 2000 2001 2002 2003 2004 2005 2006 2007
COMMIT: 2100 2101 2102 2103 2104 2105 2106 2107
device 2.2.2.2:/nfsdata/ mounted on /B with fstype nfs4 statvers=1.1
opts: rw,vers=4,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60, acdirmin=30,acdirmax=60,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys, clientaddr=3.3.3.3,minorversion=0,local_lock=none
age: 19
caps: caps=0xfff7,wtmult=512,dtsize=32768,bsize=0,namlen=255
nfsv4: bm0=0xfdffafff,bm1=0xf9be3e,acl=0x0
sec: flavor=1,pseudoflavor=1
events: 0 168232 0 0 0 10095 217808 0 2 9797 0 9739 0 0 19739 19739 0 19739 0 0 0 0 0 0 0 0 0
bytes: 1612840960 0 0 0 627536112 0 158076 0
RPC iostats version: 1.0 p/v: 100003/4 (nfs)
xprt: tcp 737 0 1 0 0 69698 69697 0 81817 0 2 1082 12119
per-op statistics
NULL: 0 0 0 0 0 0 0 0
READ: 9797 9797 0 1000 2000 71 7953 8200
WRITE: 0 0 0 0 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0
OPEN: 19740 19740 0 4737600 7343280 505 3449 4172
OPEN_CONFIRM: 10211 10211 0 1552072 694348 74 836 1008
OPEN_NOATTR: 0 0 0 0 0 0 0 0
OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0
CLOSE: 19739 19739 0 3316152 2605548 334 3045 3620
SETATTR: 0 0 0 0 0 0 0 0
FSINFO: 1 1 0 132 108 0 0 0
RENEW: 0 0 0 0 0 0 0 0
SETCLIENTID: 0 0 0 0 0 0 0 0
SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0
LOCK: 0 0 0 0 0 0 0 0
LOCKT: 0 0 0 0 0 0 0 0
LOCKU: 0 0 0 0 0 0 0 0
ACCESS: 96 96 0 14584 19584 0 8 10
GETATTR: 1 1 0 132 188 0 0 0
LOOKUP: 10095 10095 0 1655576 2382420 36 898 1072
LOOKUP_ROOT: 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0
PATHCONF: 1 1 0 128 72 0 0 0
STATFS: 0 0 0 0 0 0 0 0
READLINK: 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0
SERVER_CAPS: 2 2 0 256 176 0 0 0
DELEGRETURN: 0 0 0 0 0 0 0 0
GETACL: 0 0 0 0 0 0 0 0
SETACL: 0 0 0 0 0 0 0 0
FS_LOCATIONS: 0 0 0 0 0 0 0 0
RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0
SECINFO: 0 0 0 0 0 0 0 0
EXCHANGE_ID: 0 0 0 0 0 0 0 0
CREATE_SESSION: 0 0 0 0 0 0 0 0
DESTROY_SESSION: 500 501 502 503 504 505 506 507
SEQUENCE: 0 0 0 0 0 0 0 0
GET_LEASE_TIME: 0 0 0 0 0 0 0 0
RECLAIM_COMPLETE: 0 0 0 0 0 0 0 0
LAYOUTGET: 0 0 0 0 0 0 0 0
GETDEVICEINFO: 0 0 0 0 0 0 0 0
LAYOUTCOMMIT: 0 0 0 0 0 0 0 0
device nfsserver1:/vol/export1/bread_recipes mounted on /C with fstype nfs statvers=1.1
opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=5.4.3.2,mountvers=3,mountport=635,mountproto=udp,local_lock=none
age: 1084700
caps: caps=0x3fc7,wtmult=512,dtsize=32768,bsize=0,namlen=255
sec: flavor=1,pseudoflavor=1
events: 145712 48345501 0 2476 804 1337 49359047 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
bytes: 0 0 0 0 0 0 0 0
RPC iostats version: 1.0 p/v: 100003/3 (nfs)
xprt: tcp 871 1 1 0 0 181124336 181124308 28 1971647851 0 1100 807885669 90279840
per-op statistics
NULL: 1 2 0 44 24 0 0 0
GETATTR: 145712 145712 0 22994472 16319744 532 107480 109969
SETATTR: 0 0 0 0 0 0 0 0
LOOKUP: 2553 2553 0 385932 476148 9 1695 1739
ACCESS: 596338 596338 0 79281020 71560560 2375 228286 237993
READLINK: 0 0 0 0 0 0 0 0
READ: 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0
MKDIR: 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0
MKNOD: 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0
RMDIR: 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0
READDIR: 0 0 0 0 0 0 0 0
READDIRPLUS: 0 0 0 0 0 0 0 0
FSSTAT: 1698 1698 0 250080 285264 6 929 951
FSINFO: 34 34 0 4352 5576 0 5 5
PATHCONF: 1 1 0 128 140 0 0 0
COMMIT: 0 0 0 0 0 0 0 0
device nfsserver2:/tank/os2warp mounted on /D with fstype nfs4 statvers=1.1
opts: rw,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.66.88.239,local_lock=none
age: 2
impl_id: name='',domain='',date='0,0'
caps: caps=0xffbfff7,wtmult=512,dtsize=32768,bsize=0,namlen=255
nfsv4: bm0=0xfdffafff,bm1=0x40f9be3e,bm2=0x28803,acl=0x0,sessions,pnfs=not configured,lease_time=90,lease_expired=0
sec: flavor=1,pseudoflavor=1
events: 1 112 0 0 1 3 117 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0
bytes: 0 0 0 0 0 0 0 0
RPC iostats version: 1.1 p/v: 100003/4 (nfs)
xprt: tcp 763 0 2 0 2 39 39 0 42 0 2 0 3
per-op statistics
NULL: 1 1 0 44 24 0 0 1 0
READ: 0 0 0 0 0 0 0 0 0
WRITE: 0 0 0 0 0 0 0 0 0
COMMIT: 0 0 0 0 0 0 0 0 0
OPEN: 0 0 0 0 0 0 0 0 0
OPEN_CONFIRM: 0 0 0 0 0 0 0 0 0
OPEN_NOATTR: 0 0 0 0 0 0 0 0 0
OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0 0
CLOSE: 0 0 0 0 0 0 0 0 0
SETATTR: 0 0 0 0 0 0 0 0 0
FSINFO: 1 1 0 168 164 0 0 0 0
RENEW: 0 0 0 0 0 0 0 0 0
SETCLIENTID: 0 0 0 0 0 0 0 0 0
SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0 0
LOCK: 0 0 0 0 0 0 0 0 0
LOCKT: 0 0 0 0 0 0 0 0 0
LOCKU: 0 0 0 0 0 0 0 0 0
ACCESS: 3 3 0 600 504 0 1 1 0
GETATTR: 2 2 0 364 480 0 1 1 0
LOOKUP: 3 3 0 628 484 0 1 1 2
LOOKUP_ROOT: 0 0 0 0 0 0 0 0 0
REMOVE: 0 0 0 0 0 0 0 0 0
RENAME: 0 0 0 0 0 0 0 0 0
LINK: 0 0 0 0 0 0 0 0 0
SYMLINK: 0 0 0 0 0 0 0 0 0
CREATE: 0 0 0 0 0 0 0 0 0
PATHCONF: 1 1 0 160 116 0 0 0 0
STATFS: 1 1 0 164 160 0 0 0 0
READLINK: 0 0 0 0 0 0 0 0 0
READDIR: 1 1 0 224 11968 0 1 1 0
SERVER_CAPS: 2 2 0 336 328 0 1 1 0
DELEGRETURN: 0 0 0 0 0 0 0 0 0
GETACL: 0 0 0 0 0 0 0 0 0
SETACL: 0 0 0 0 0 0 0 0 0
FS_LOCATIONS: 0 0 0 0 0 0 0 0 0
RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0 0
SECINFO: 0 0 0 0 0 0 0 0 0
FSID_PRESENT: 0 0 0 0 0 0 0 0 0
EXCHANGE_ID: 2 2 0 480 200 0 2 2 0
CREATE_SESSION: 1 1 0 200 124 0 0 0 0
DESTROY_SESSION: 0 0 0 0 0 0 0 0 0
SEQUENCE: 0 0 0 0 0 0 0 0 0
GET_LEASE_TIME: 0 0 0 0 0 0 0 0 0
RECLAIM_COMPLETE: 1 1 0 128 88 0 107 107 0
LAYOUTGET: 0 0 0 0 0 0 0 0 0
GETDEVICEINFO: 0 0 0 0 0 0 0 0 0
LAYOUTCOMMIT: 0 0 0 0 0 0 0 0 0
LAYOUTRETURN: 0 0 0 0 0 0 0 0 0
SECINFO_NO_NAME: 0 0 0 0 0 0 0 0 0
TEST_STATEID: 0 0 0 0 0 0 0 0 0
FREE_STATEID: 0 0 0 0 0 0 0 0 0
GETDEVICELIST: 0 0 0 0 0 0 0 0 0
BIND_CONN_TO_SESSION: 0 0 0 0 0 0 0 0 0
DESTROY_CLIENTID: 0 0 0 0 0 0 0 0 0
SEEK: 0 0 0 0 0 0 0 0 0
ALLOCATE: 0 0 0 0 0 0 0 0 0
DEALLOCATE: 0 0 0 0 0 0 0 0 0
LAYOUTSTATS: 0 0 0 0 0 0 0 0 0
CLONE: 0 0 0 0 0 0 0 0 0
COPY: 0 0 0 0 0 0 0 0 0
OFFLOAD_CANCEL: 0 0 0 0 0 0 0 0 0
LOOKUPP: 0 0 0 0 0 0 0 0 0
LAYOUTERROR: 0 0 0 0 0 0 0 0 0
COPY_NOTIFY: 0 0 0 0 0 0 0 0 0
GETXATTR: 0 0 0 0 0 0 0 0 0
SETXATTR: 0 0 0 0 0 0 0 0 0
LISTXATTRS: 0 0 0 0 0 0 0 0 0
REMOVEXATTR: 0 0 0 0 0 0 0 0 0
LAYOUTRETURN: 0 0 0 0 0 0 0 0