diff --git a/README.md b/README.md index b1cf1ecf4..59aa7672c 100644 --- a/README.md +++ b/README.md @@ -259,6 +259,7 @@ For documentation on the latest development code see the [documentation index][d * [net](./plugins/inputs/net) * [net_response](./plugins/inputs/net_response) * [netstat](./plugins/inputs/net) +* [nfsclient](./plugins/inputs/nfsclient) * [nginx](./plugins/inputs/nginx) * [nginx_plus_api](./plugins/inputs/nginx_plus_api) * [nginx_plus](./plugins/inputs/nginx_plus) diff --git a/plugins/inputs/all/all.go b/plugins/inputs/all/all.go index 65d8d5254..d5eeead0a 100644 --- a/plugins/inputs/all/all.go +++ b/plugins/inputs/all/all.go @@ -112,6 +112,7 @@ import ( _ "github.com/influxdata/telegraf/plugins/inputs/neptune_apex" _ "github.com/influxdata/telegraf/plugins/inputs/net" _ "github.com/influxdata/telegraf/plugins/inputs/net_response" + _ "github.com/influxdata/telegraf/plugins/inputs/nfsclient" _ "github.com/influxdata/telegraf/plugins/inputs/nginx" _ "github.com/influxdata/telegraf/plugins/inputs/nginx_plus" _ "github.com/influxdata/telegraf/plugins/inputs/nginx_plus_api" diff --git a/plugins/inputs/nfsclient/README.md b/plugins/inputs/nfsclient/README.md new file mode 100644 index 000000000..965bcc524 --- /dev/null +++ b/plugins/inputs/nfsclient/README.md @@ -0,0 +1,181 @@ +#### Description + +The NFSClient plugin collects data from /proc/self/mountstats. By default, only a limited number of general system-level metrics are collected, including basic read/write counts. +If `fullstat` is set, a great deal of additional metrics are collected, detailed below. + +**NOTE** Many of the metrics, even if tagged with a mount point, are really _per-server_. Thus, if you mount these two shares: `nfs01:/vol/foo/bar` and `nfs01:/vol/foo/baz`, there will be two near identical entries in /proc/self/mountstats. This is a limitation of the metrics exposed by the kernel, not the telegraf plugin. + +#### Plugin arguments: +- **fullstat** bool: Collect per-operation type metrics. Defaults to false. +- **include_mounts** list(string): gather metrics for only these mounts. Default is to watch all mounts. +- **exclude_mounts** list(string): gather metrics for all mounts, except those listed in this option. Excludes take precedence over includes. +- **include_operations** list(string): List of specific NFS operations to track. See /proc/self/mountstats (the "per-op statistics" section) for complete lists of valid options for NFSv3 and NFSV4. The default is to gather all metrics, but this is almost certainly *not* what you want (there are 22 operations for NFSv3, and well over 50 for NFSv4). A suggested 'minimal' list of operations to collect for basic usage: `['READ','WRITE','ACCESS','GETATTR','READDIR','LOOKUP','LOOKUP']` +- **exclude_operations** list(string): Gather all metrics, except those listed. Excludes take precedence over includes. + +*N.B.* the `include_mounts` and `exclude_mounts` arguments are both applied to the local mount location (e.g. /mnt/NFS), not the server export (e.g. nfsserver:/vol/NFS). Go regexp patterns can be used in either. + +#### Examples + +```toml +[[inputs.nfsclient]] + ## Read more low-level metrics (optional, defaults to false) + # fullstat = false + + ## List of mounts to explictly include or exclude (optional) + ## The pattern (Go regexp) is matched against the mount point (not the + ## device being mounted). If include_mounts is set, all mounts are ignored + ## unless present in the list. If a mount is listed in both include_mounts + ## and exclude_mounts, it is excluded. Go regexp patterns can be used. + # include_mounts = [] + # exclude_mounts = [] + + ## List of operations to include or exclude from collecting. This applies + ## only when fullstat=true. Symantics are similar to {include,exclude}_mounts: + ## the default is to collect everything; when include_operations is set, only + ## those OPs are collected; when exclude_operations is set, all are collected + ## except those listed. If include and exclude are set, the OP is excluded. + ## See /proc/self/mountstats for a list of valid operations; note that + ## NFSv3 and NFSv4 have different lists. While it is not possible to + ## have different include/exclude lists for NFSv3/4, unused elements + ## in the list should be okay. It is possible to have different lists + ## for different mountpoints: use mulitple [[input.nfsclient]] stanzas, + ## with their own lists. See "include_mounts" above, and be careful of + ## duplicate metrics. + # include_operations = [] + # exclude_operations = [] +``` + +Example output for basic metrics showing server-wise read and write data: + +``` +nfsstat,mountpoint=/NFS,operation=READ,serverexport=1.2.3.4:/storage/NFS ops=600i,retrans=1i,bytes=1207i,rtt=606i,exe=607i 1612651512000000000 +nfsstat,mountpoint=/NFS,operation=WRITE,serverexport=1.2.3.4:/storage/NFS bytes=1407i,rtt=706i,exe=707i,ops=700i,retrans=1i 1612651512000000000 + +``` + +Example output for `fullstat=true` metrics, which includes additional measurements for `nfs_bytes`, `nfs_events`, and `nfs_xprt_tcp` (and `nfs_xprt_udp` if present). +Additionally, per-OP metrics are collected, with examples for READ, LOOKUP, and NULL shown. +Please refer to `/proc/self/mountstats` for a list of supported NFS operations, as it changes as it changes periodically. + +``` +nfs_bytes,mountpoint=/home,serverexport=nfs01:/vol/home directreadbytes=0i,directwritebytes=0i,normalreadbytes=42648757667i,normalwritebytes=0i,readpages=10404603i,serverreadbytes=42617098139i,serverwritebytes=0i,writepages=0i 1608787697000000000 +nfs_events,mountpoint=/home,serverexport=nfs01:/vol/home attrinvalidates=116i,congestionwait=0i,datainvalidates=65i,delay=0i,dentryrevalidates=5911243i,extendwrite=0i,inoderevalidates=200378i,pnfsreads=0i,pnfswrites=0i,setattrtrunc=0i,shortreads=0i,shortwrites=0i,sillyrenames=0i,vfsaccess=7203852i,vfsflush=117405i,vfsfsync=0i,vfsgetdents=3368i,vfslock=0i,vfslookup=740i,vfsopen=157281i,vfsreadpage=16i,vfsreadpages=86874i,vfsrelease=155526i,vfssetattr=0i,vfsupdatepage=0i,vfswritepage=0i,vfswritepages=215514i 1608787697000000000 +nfs_xprt_tcp,mountpoint=/home,serverexport=nfs01:/vol/home backlogutil=0i,badxids=0i,bind_count=1i,connect_count=1i,connect_time=0i,idle_time=0i,inflightsends=15659826i,rpcreceives=2173896i,rpcsends=2173896i 1608787697000000000 + +nfs_ops,mountpoint=/NFS,operation=NULL,serverexport=1.2.3.4:/storage/NFS trans=0i,timeouts=0i,bytes_sent=0i,bytes_recv=0i,queue_time=0i,response_time=0i,total_time=0i,ops=0i 1612651512000000000 +nfs_ops,mountpoint=/NFS,operation=READ,serverexport=1.2.3.4:/storage/NFS bytes=1207i,timeouts=602i,total_time=607i,exe=607i,trans=601i,bytes_sent=603i,bytes_recv=604i,queue_time=605i,ops=600i,retrans=1i,rtt=606i,response_time=606i 1612651512000000000 +nfs_ops,mountpoint=/NFS,operation=WRITE,serverexport=1.2.3.4:/storage/NFS ops=700i,bytes=1407i,exe=707i,trans=701i,timeouts=702i,response_time=706i,total_time=707i,retrans=1i,rtt=706i,bytes_sent=703i,bytes_recv=704i,queue_time=705i 1612651512000000000 +``` + +#### References +1. [nfsiostat](http://git.linux-nfs.org/?p=steved/nfs-utils.git;a=summary) +2. [net/sunrpc/stats.c - Linux source code](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/sunrpc/stats.c) +3. [What is in /proc/self/mountstats for NFS mounts: an introduction](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsIndex) +4. [The xprt: data for NFS mounts in /proc/self/mountstats](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsXprt) + + + +#### Measurements & Fields + +Always collected: + +- nfsstat + - bytes (integer, bytes) - The total number of bytes exchanged doing this operation. This is bytes sent *and* received, including overhead *and* payload. (bytes = OP_bytes_sent + OP_bytes_recv. See nfs_ops below) + - ops (integer, count) - The number of operations of this type executed. + - retrans (integer, count) - The number of times an operation had to be retried (retrans = OP_trans - OP_ops. See nfs_ops below) + - exe (integer, miliseconds) - The number of miliseconds it took to process the operations. + - rtt (integer, miliseconds) - The round-trip time for operations. + +In addition enabling `fullstat` will make many more metrics available. + +#### Tags + +- All measurements have the following tags: + - mountpoint - The local mountpoint, for instance: "/var/www" + - serverexport - The full server export, for instance: "nfsserver.example.org:/export" + +- Measurements nfsstat and nfs_ops will also include: + - operation - the NFS operation in question. `READ` or `WRITE` for nfsstat, but potentially one of ~20 or ~50, depending on NFS version. A complete list of operations supported is visible in `/proc/self/mountstats`. + + + +### Additional metrics + +When `fullstat` is true, additional measurements are collected. Tags are the same as above. + +#### NFS Operations + +Most descriptions come from Reference [[3](https://utcc.utoronto.ca/~cks/space/blog/linux/NFSMountstatsIndex)] and `nfs_iostat.h`. Field order and names are the same as in `/proc/self/mountstats` and the Kernel source. + +Please refer to `/proc/self/mountstats` for a list of supported NFS operations, as it changes occasionally. + +- nfs_bytes + - fields: + - normalreadbytes - (int, bytes) - Bytes read from the server via `read()` + - normalwritebytes - (int, bytes) - Bytes written to the server via `write()` + - directreadbytes - (int, bytes) - Bytes read with O_DIRECT set + - directwritebytes - (int, bytes) -Bytes written with O_DIRECT set + - serverreadbytes - (int, bytes) - Bytes read via NFS READ (via `mmap()`) + - serverwritebytes - (int, bytes) - Bytes written via NFS WRITE (via `mmap()`) + - readpages - (int, count) - Number of pages read + - writepages - (int, count) - Number of pages written + +- nfs_events - Per-event metrics + - fields: + - inoderevalidates - (int, count) - How many times cached inode attributes have to be re-validated from the server. + - dentryrevalidates - (int, count) - How many times cached dentry nodes have to be re-validated. + - datainvalidates - (int, count) - How many times an inode had its cached data thrown out. + - attrinvalidates - (int, count) - How many times an inode has had cached inode attributes invalidated. + - vfsopen - (int, count) - How many times files or directories have been `open()`'d. + - vfslookup - (int, count) - How many name lookups in directories there have been. + - vfsaccess - (int, count) - Number of calls to `access()`. (formerly called "vfspermission") + + - vfsupdatepage - (int, count) - Count of updates (and potential writes) to pages. + - vfsreadpage - (int, count) - Number of pages read. + - vfsreadpages - (int, count) - Count of how many times a _group_ of pages was read (possibly via `mmap()`?). + - vfswritepage - (int, count) - Number of pages written. + - vfswritepages - (int, count) - Count of how many times a _group_ of pages was written (possibly via `mmap()`?) + - vfsgetdents - (int, count) - Count of directory entry reads with getdents(). These reads can be served from cache and don't necessarily imply actual NFS requests. (formerly called "vfsreaddir") + - vfssetattr - (int, count) - How many times we've set attributes on inodes. + - vfsflush - (int, count) - Count of times pending writes have been forcibly flushed to the server. + - vfsfsync - (int, count) - Count of calls to `fsync()` on directories and files. + - vfslock - (int, count) - Number of times a lock was attempted on a file (regardless of success or not). + - vfsrelease - (int, count) - Number of calls to `close()`. + - congestionwait - (int, count) - Believe unused by the Linux kernel, but it is part of the NFS spec. + - setattrtrunc - (int, count) - How many times files have had their size truncated. + - extendwrite - (int, count) - How many times a file has been grown because you're writing beyond the existing end of the file. + - sillyrenames - (int, count) - Number of times an in-use file was removed (thus creating a temporary ".nfsXXXXXX" file) + - shortreads - (int, count) - Number of times the NFS server returned less data than requested. + - shortwrites - (int, count) - Number of times NFS server reports it wrote less data than requested. + - delay - (int, count) - Occurances of EJUKEBOX ("Jukebox Delay", probably unused) + - pnfsreads - (int, count) - Count of NFS v4.1+ pNFS reads. + - pnfswrites - (int, count) - Count of NFS v4.1+ pNFS writes. + + - nfs_xprt_tcp + - fields: + - bind_count - (int, count) - Number of _completely new_ mounts to this server (sometimes 0?) + - connect_count - (int, count) - How many times the client has connected to the server in question + - connect_time - (int, jiffies) - How long the NFS client has spent waiting for its connection(s) to the server to be established. + - idle_time - (int, seconds) - How long (in seconds) since the NFS mount saw any RPC traffic. + - rpcsends - (int, count) - How many RPC requests this mount has sent to the server. + - rpcreceives - (int, count) - How many RPC replies this mount has received from the server. + - badxids - (int, count) - Count of XIDs sent by the server that the client doesn't know about. + - inflightsends - (int, count) - Number of outstanding requests; always >1. (See reference #4 for comment on this field) + - backlogutil - (int, count) - Cumulative backlog count + +- nfs_xprt_udp + - fields: + - [same as nfs_xprt_tcp, except for connect_count, connect_time, and idle_time] + +- nfs_ops + - fields (In all cases, the `operations` tag is set to the uppercase name of the NFS operation, _e.g._ "READ", "FSINFO", _etc_. See /proc/self/mountstats for a full list): + - ops - (int, count) - Total operations of this type. + - trans - (int, count) - Total transmissions of this type, including retransmissions: `OP_ops - OP_trans = total_retransmissions` (lower is better). + - timeouts - (int, count) - Number of major timeouts. + - bytes_sent - (int, count) - Bytes received, including headers (should also be close to on-wire size). + - bytes_recv - (int, count) - Bytes sent, including headers (should be close to on-wire size). + - queue_time - (int, milliseconds) - Cumulative time a request waited in the queue before sending this OP type. + - response_time - (int, milliseconds) - Cumulative time waiting for a response for this OP type. + - total_time - (int, milliseconds) - Cumulative time a request waited in the queue before sending. + - errors - (int, count) - Total number operations that complete with tk_status < 0 (usually errors). This is a new field, present in kernel >=5.3, mountstats version 1.1 + diff --git a/plugins/inputs/nfsclient/nfsclient.go b/plugins/inputs/nfsclient/nfsclient.go new file mode 100644 index 000000000..37fa64fef --- /dev/null +++ b/plugins/inputs/nfsclient/nfsclient.go @@ -0,0 +1,497 @@ +package nfsclient + +import ( + "bufio" + "log" + "os" + "regexp" + "strconv" + "strings" + + "github.com/influxdata/telegraf" + "github.com/influxdata/telegraf/internal/choice" + "github.com/influxdata/telegraf/plugins/inputs" +) + +type NFSClient struct { + Fullstat bool `toml:"fullstat"` + IncludeMounts []string `toml:"include_mounts"` + ExcludeMounts []string `toml:"exclude_mounts"` + IncludeOperations []string `toml:"include_operations"` + ExcludeOperations []string `toml:"exclude_operations"` + Log telegraf.Logger `toml:"-"` + nfs3Ops map[string]bool + nfs4Ops map[string]bool + mountstatsPath string +} + +const sampleConfig = ` + ## Read more low-level metrics (optional, defaults to false) + # fullstat = false + + ## List of mounts to explictly include or exclude (optional) + ## The pattern (Go regexp) is matched against the mount point (not the + ## device being mounted). If include_mounts is set, all mounts are ignored + ## unless present in the list. If a mount is listed in both include_mounts + ## and exclude_mounts, it is excluded. Go regexp patterns can be used. + # include_mounts = [] + # exclude_mounts = [] + + ## List of operations to include or exclude from collecting. This applies + ## only when fullstat=true. Symantics are similar to {include,exclude}_mounts: + ## the default is to collect everything; when include_operations is set, only + ## those OPs are collected; when exclude_operations is set, all are collected + ## except those listed. If include and exclude are set, the OP is excluded. + ## See /proc/self/mountstats for a list of valid operations; note that + ## NFSv3 and NFSv4 have different lists. While it is not possible to + ## have different include/exclude lists for NFSv3/4, unused elements + ## in the list should be okay. It is possible to have different lists + ## for different mountpoints: use mulitple [[input.nfsclient]] stanzas, + ## with their own lists. See "include_mounts" above, and be careful of + ## duplicate metrics. + # include_operations = [] + # exclude_operations = [] +` + +func (n *NFSClient) SampleConfig() string { + return sampleConfig +} + +func (n *NFSClient) Description() string { + return "Read per-mount NFS client metrics from /proc/self/mountstats" +} + +func convertToInt64(line []string) []int64 { + /* A "line" of input data (a pre-split array of strings) is + processed one field at a time. Each field is converted to + an int64 value, and appened to an array of return values. + On an error, check for ErrRange, and throw a fatal error + if found. This situation indicates a pretty major issue in + the /proc/self/mountstats file, and returning faulty data + is worse than no data. Other errors are ignored, and append + whatever we got in the first place (probably 0). + Yes, this is ugly. */ + + var nline []int64 + + if len(line) < 2 { + return nline + } + + // Skip the first field; it's handled specially as the "first" variable + for _, l := range line[1:] { + val, err := strconv.ParseInt(l, 10, 64) + if err != nil { + if numError, ok := err.(*strconv.NumError); ok { + if numError.Err == strconv.ErrRange { + log.Fatalf("ErrRange: line:[%v] raw:[%v] -> parsed:[%v]\n", line, l, val) + } + } + } + nline = append(nline, val) + } + return nline +} + +func (n *NFSClient) parseStat(mountpoint string, export string, version string, line []string, fullstat bool, acc telegraf.Accumulator) error { + tags := map[string]string{"mountpoint": mountpoint, "serverexport": export} + nline := convertToInt64(line) + + if len(nline) == 0 { + n.Log.Warnf("Parsing Stat line with one field: %s\n", line) + return nil + } + + first := strings.Replace(line[0], ":", "", 1) + + var eventsFields = []string{ + "inoderevalidates", + "dentryrevalidates", + "datainvalidates", + "attrinvalidates", + "vfsopen", + "vfslookup", + "vfsaccess", + "vfsupdatepage", + "vfsreadpage", + "vfsreadpages", + "vfswritepage", + "vfswritepages", + "vfsgetdents", + "vfssetattr", + "vfsflush", + "vfsfsync", + "vfslock", + "vfsrelease", + "congestionwait", + "setattrtrunc", + "extendwrite", + "sillyrenames", + "shortreads", + "shortwrites", + "delay", + "pnfsreads", + "pnfswrites", + } + + var bytesFields = []string{ + "normalreadbytes", + "normalwritebytes", + "directreadbytes", + "directwritebytes", + "serverreadbytes", + "serverwritebytes", + "readpages", + "writepages", + } + + var xprtudpFields = []string{ + "bind_count", + "rpcsends", + "rpcreceives", + "badxids", + "inflightsends", + "backlogutil", + } + + var xprttcpFields = []string{ + "bind_count", + "connect_count", + "connect_time", + "idle_time", + "rpcsends", + "rpcreceives", + "badxids", + "inflightsends", + "backlogutil", + } + + var nfsopFields = []string{ + "ops", + "trans", + "timeouts", + "bytes_sent", + "bytes_recv", + "queue_time", + "response_time", + "total_time", + "errors", + } + + var fields = make(map[string]interface{}) + + switch first { + case "READ", "WRITE": + fields["ops"] = nline[0] + fields["retrans"] = nline[1] - nline[0] + fields["bytes"] = nline[3] + nline[4] + fields["rtt"] = nline[6] + fields["exe"] = nline[7] + tags["operation"] = first + acc.AddFields("nfsstat", fields, tags) + } + + if fullstat { + switch first { + case "events": + if len(nline) >= len(eventsFields) { + for i, t := range eventsFields { + fields[t] = nline[i] + } + acc.AddFields("nfs_events", fields, tags) + } + + case "bytes": + if len(nline) >= len(bytesFields) { + for i, t := range bytesFields { + fields[t] = nline[i] + } + acc.AddFields("nfs_bytes", fields, tags) + } + + case "xprt": + if len(line) > 1 { + switch line[1] { + case "tcp": + if len(nline)+2 >= len(xprttcpFields) { + for i, t := range xprttcpFields { + fields[t] = nline[i+2] + } + acc.AddFields("nfs_xprt_tcp", fields, tags) + } + case "udp": + if len(nline)+2 >= len(xprtudpFields) { + for i, t := range xprtudpFields { + fields[t] = nline[i+2] + } + acc.AddFields("nfs_xprt_udp", fields, tags) + } + } + } + } + + if (version == "3" && n.nfs3Ops[first]) || (version == "4" && n.nfs4Ops[first]) { + tags["operation"] = first + if len(nline) <= len(nfsopFields) { + for i, t := range nline { + fields[nfsopFields[i]] = t + } + acc.AddFields("nfs_ops", fields, tags) + } + } + + } + + return nil +} + +func (n *NFSClient) processText(scanner *bufio.Scanner, acc telegraf.Accumulator) error { + var mount string + var version string + var export string + var skip bool + + for scanner.Scan() { + line := strings.Fields(scanner.Text()) + + line_len := len(line) + + if line_len == 0 { + continue + } + + skip = false + + // This denotes a new mount has been found, so set + // mount and export, and stop skipping (for now) + if line_len > 4 && choice.Contains("fstype", line) && (choice.Contains("nfs", line) || choice.Contains("nfs4", line)) { + mount = line[4] + export = line[1] + } else if line_len > 5 && (choice.Contains("(nfs)", line) || choice.Contains("(nfs4)", line)) { + version = strings.Split(line[5], "/")[1] + } + + if mount == "" { + continue + } + + if len(n.IncludeMounts) > 0 { + skip = true + for _, RE := range n.IncludeMounts { + matched, _ := regexp.MatchString(RE, mount) + if matched { + skip = false + break + } + } + } + + if !skip && len(n.ExcludeMounts) > 0 { + for _, RE := range n.ExcludeMounts { + matched, _ := regexp.MatchString(RE, mount) + if matched { + skip = true + break + } + } + } + + if !skip { + n.parseStat(mount, export, version, line, n.Fullstat, acc) + } + } + return nil +} + +func (n *NFSClient) getMountStatsPath() string { + + path := "/proc/self/mountstats" + if os.Getenv("MOUNT_PROC") != "" { + path = os.Getenv("MOUNT_PROC") + } + n.Log.Debugf("using [%s] for mountstats", path) + return path +} + +func (n *NFSClient) Gather(acc telegraf.Accumulator) error { + + file, err := os.Open(n.mountstatsPath) + if err != nil { + n.Log.Errorf("Failed opening the [%s] file: %s ", file, err) + return err + } + defer file.Close() + + scanner := bufio.NewScanner(file) + n.processText(scanner, acc) + + if err := scanner.Err(); err != nil { + n.Log.Errorf("%s", err) + return err + } + + return nil +} + +func (n *NFSClient) Init() error { + + var nfs3Fields = []string{ + "NULL", + "GETATTR", + "SETATTR", + "LOOKUP", + "ACCESS", + "READLINK", + "READ", + "WRITE", + "CREATE", + "MKDIR", + "SYMLINK", + "MKNOD", + "REMOVE", + "RMDIR", + "RENAME", + "LINK", + "READDIR", + "READDIRPLUS", + "FSSTAT", + "FSINFO", + "PATHCONF", + "COMMIT", + } + + var nfs4Fields = []string{ + "NULL", + "READ", + "WRITE", + "COMMIT", + "OPEN", + "OPEN_CONFIRM", + "OPEN_NOATTR", + "OPEN_DOWNGRADE", + "CLOSE", + "SETATTR", + "FSINFO", + "RENEW", + "SETCLIENTID", + "SETCLIENTID_CONFIRM", + "LOCK", + "LOCKT", + "LOCKU", + "ACCESS", + "GETATTR", + "LOOKUP", + "LOOKUP_ROOT", + "REMOVE", + "RENAME", + "LINK", + "SYMLINK", + "CREATE", + "PATHCONF", + "STATFS", + "READLINK", + "READDIR", + "SERVER_CAPS", + "DELEGRETURN", + "GETACL", + "SETACL", + "FS_LOCATIONS", + "RELEASE_LOCKOWNER", + "SECINFO", + "FSID_PRESENT", + "EXCHANGE_ID", + "CREATE_SESSION", + "DESTROY_SESSION", + "SEQUENCE", + "GET_LEASE_TIME", + "RECLAIM_COMPLETE", + "LAYOUTGET", + "GETDEVICEINFO", + "LAYOUTCOMMIT", + "LAYOUTRETURN", + "SECINFO_NO_NAME", + "TEST_STATEID", + "FREE_STATEID", + "GETDEVICELIST", + "BIND_CONN_TO_SESSION", + "DESTROY_CLIENTID", + "SEEK", + "ALLOCATE", + "DEALLOCATE", + "LAYOUTSTATS", + "CLONE", + "COPY", + "OFFLOAD_CANCEL", + "LOOKUPP", + "LAYOUTERROR", + "COPY_NOTIFY", + "GETXATTR", + "SETXATTR", + "LISTXATTRS", + "REMOVEXATTR", + } + + nfs3Ops := make(map[string]bool) + nfs4Ops := make(map[string]bool) + + n.mountstatsPath = n.getMountStatsPath() + + if len(n.IncludeOperations) == 0 { + for _, Op := range nfs3Fields { + nfs3Ops[Op] = true + } + for _, Op := range nfs4Fields { + nfs4Ops[Op] = true + } + } else { + for _, Op := range n.IncludeOperations { + nfs3Ops[Op] = true + } + for _, Op := range n.IncludeOperations { + nfs4Ops[Op] = true + } + } + + if len(n.ExcludeOperations) > 0 { + for _, Op := range n.ExcludeOperations { + if nfs3Ops[Op] { + delete(nfs3Ops, Op) + } + if nfs4Ops[Op] { + delete(nfs4Ops, Op) + } + } + } + + if len(n.IncludeMounts) > 0 { + n.Log.Debugf("Including these mount patterns: %v", n.IncludeMounts) + } else { + n.Log.Debugf("Including all mounts.") + } + + if len(n.ExcludeMounts) > 0 { + n.Log.Debugf("Excluding these mount patterns: %v", n.ExcludeMounts) + } else { + n.Log.Debugf("Not excluding any mounts.") + } + + if len(n.IncludeOperations) > 0 { + n.Log.Debugf("Including these operations: %v", n.IncludeOperations) + } else { + n.Log.Debugf("Including all operations.") + } + + if len(n.ExcludeOperations) > 0 { + n.Log.Debugf("Excluding these mount patterns: %v", n.ExcludeOperations) + } else { + n.Log.Debugf("Not excluding any operations.") + } + + return nil +} + +func init() { + inputs.Add("nfsclient", func() telegraf.Input { + return &NFSClient{} + }) +} diff --git a/plugins/inputs/nfsclient/nfsclient_test.go b/plugins/inputs/nfsclient/nfsclient_test.go new file mode 100644 index 000000000..f4f008fbc --- /dev/null +++ b/plugins/inputs/nfsclient/nfsclient_test.go @@ -0,0 +1,177 @@ +package nfsclient + +import ( + "bufio" + "github.com/influxdata/telegraf/testutil" + "os" + "strings" + "testing" +) + +func getMountStatsPath() string { + + path := "./testdata/mountstats" + if os.Getenv("MOUNT_PROC") != "" { + path = os.Getenv("MOUNT_PROC") + } + + return path +} + +func TestNFSClientParsev3(t *testing.T) { + var acc testutil.Accumulator + + nfsclient := NFSClient{} + nfsclient.nfs3Ops = map[string]bool{"READLINK": true, "GETATTR": false} + nfsclient.nfs4Ops = map[string]bool{"READLINK": true, "GETATTR": false} + data := strings.Fields(" READLINK: 500 501 502 503 504 505 506 507") + nfsclient.parseStat("1.2.3.4:/storage/NFS", "/A", "3", data, true, &acc) + + fields_ops := map[string]interface{}{ + "ops": int64(500), + "trans": int64(501), + "timeouts": int64(502), + "bytes_sent": int64(503), + "bytes_recv": int64(504), + "queue_time": int64(505), + "response_time": int64(506), + "total_time": int64(507), + } + acc.AssertContainsFields(t, "nfs_ops", fields_ops) +} + +func TestNFSClientParsev4(t *testing.T) { + var acc testutil.Accumulator + + nfsclient := NFSClient{} + nfsclient.nfs3Ops = map[string]bool{"DESTROY_SESSION": true, "GETATTR": false} + nfsclient.nfs4Ops = map[string]bool{"DESTROY_SESSION": true, "GETATTR": false} + data := strings.Fields(" DESTROY_SESSION: 500 501 502 503 504 505 506 507") + nfsclient.parseStat("2.2.2.2:/nfsdata/", "/B", "4", data, true, &acc) + + fields_ops := map[string]interface{}{ + "ops": int64(500), + "trans": int64(501), + "timeouts": int64(502), + "bytes_sent": int64(503), + "bytes_recv": int64(504), + "queue_time": int64(505), + "response_time": int64(506), + "total_time": int64(507), + } + acc.AssertContainsFields(t, "nfs_ops", fields_ops) +} + +func TestNFSClientProcessStat(t *testing.T) { + var acc testutil.Accumulator + + nfsclient := NFSClient{} + nfsclient.Fullstat = false + + file, _ := os.Open(getMountStatsPath()) + defer file.Close() + + scanner := bufio.NewScanner(file) + + nfsclient.processText(scanner, &acc) + + fields_readstat := map[string]interface{}{ + "ops": int64(600), + "retrans": int64(1), + "bytes": int64(1207), + "rtt": int64(606), + "exe": int64(607), + } + + read_tags := map[string]string{ + "serverexport": "1.2.3.4:/storage/NFS", + "mountpoint": "/A", + "operation": "READ", + } + + acc.AssertContainsTaggedFields(t, "nfsstat", fields_readstat, read_tags) + + fields_writestat := map[string]interface{}{ + "ops": int64(700), + "retrans": int64(1), + "bytes": int64(1407), + "rtt": int64(706), + "exe": int64(707), + } + + write_tags := map[string]string{ + "serverexport": "1.2.3.4:/storage/NFS", + "mountpoint": "/A", + "operation": "WRITE", + } + acc.AssertContainsTaggedFields(t, "nfsstat", fields_writestat, write_tags) +} + +func TestNFSClientProcessFull(t *testing.T) { + var acc testutil.Accumulator + + nfsclient := NFSClient{} + nfsclient.Fullstat = true + + file, _ := os.Open(getMountStatsPath()) + defer file.Close() + + scanner := bufio.NewScanner(file) + + nfsclient.processText(scanner, &acc) + + fields_events := map[string]interface{}{ + "inoderevalidates": int64(301736), + "dentryrevalidates": int64(22838), + "datainvalidates": int64(410979), + "attrinvalidates": int64(26188427), + "vfsopen": int64(27525), + "vfslookup": int64(9140), + "vfsaccess": int64(114420), + "vfsupdatepage": int64(30785253), + "vfsreadpage": int64(5308856), + "vfsreadpages": int64(5364858), + "vfswritepage": int64(30784819), + "vfswritepages": int64(79832668), + "vfsgetdents": int64(170), + "vfssetattr": int64(64), + "vfsflush": int64(18194), + "vfsfsync": int64(29294718), + "vfslock": int64(0), + "vfsrelease": int64(18279), + "congestionwait": int64(0), + "setattrtrunc": int64(2), + "extendwrite": int64(785551), + "sillyrenames": int64(0), + "shortreads": int64(0), + "shortwrites": int64(0), + "delay": int64(0), + "pnfsreads": int64(0), + "pnfswrites": int64(0), + } + fields_bytes := map[string]interface{}{ + "normalreadbytes": int64(204440464584), + "normalwritebytes": int64(110857586443), + "directreadbytes": int64(783170354688), + "directwritebytes": int64(296174954496), + "serverreadbytes": int64(1134399088816), + "serverwritebytes": int64(407107155723), + "readpages": int64(85749323), + "writepages": int64(30784819), + } + fields_xprt_tcp := map[string]interface{}{ + "bind_count": int64(1), + "connect_count": int64(1), + "connect_time": int64(0), + "idle_time": int64(0), + "rpcsends": int64(96172963), + "rpcreceives": int64(96172963), + "badxids": int64(0), + "inflightsends": int64(620878754), + "backlogutil": int64(0), + } + + acc.AssertContainsFields(t, "nfs_events", fields_events) + acc.AssertContainsFields(t, "nfs_bytes", fields_bytes) + acc.AssertContainsFields(t, "nfs_xprt_tcp", fields_xprt_tcp) +} diff --git a/plugins/inputs/nfsclient/testdata/mountstats b/plugins/inputs/nfsclient/testdata/mountstats new file mode 100644 index 000000000..86651d20d --- /dev/null +++ b/plugins/inputs/nfsclient/testdata/mountstats @@ -0,0 +1,231 @@ +device rootfs mounted on / with fstype rootfs +device proc mounted on /proc with fstype proc +device sysfs mounted on /sys with fstype sysfs +device devtmpfs mounted on /dev with fstype devtmpfs +device devpts mounted on /dev/pts with fstype devpts +device tmpfs mounted on /dev/shm with fstype tmpfs +device /dev/loop0 mounted on /dev/.initramfs/live with fstype iso9660 +device /dev/loop6 mounted on / with fstype ext4 +device /proc/bus/usb mounted on /proc/bus/usb with fstype usbfs +device none mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc +device /tmp mounted on /tmp with fstype tmpfs +device /home mounted on /home with fstype tmpfs +device /var mounted on /var with fstype tmpfs +device /etc mounted on /etc with fstype tmpfs +device /dev/ram1 mounted on /root with fstype ext2 +device cgroup mounted on /cgroup/cpuset with fstype cgroup +device cgroup mounted on /cgroup/cpu with fstype cgroup +device cgroup mounted on /cgroup/cpuacct with fstype cgroup +device cgroup mounted on /cgroup/memory with fstype cgroup +device cgroup mounted on /cgroup/devices with fstype cgroup +device cgroup mounted on /cgroup/freezer with fstype cgroup +device cgroup mounted on /cgroup/net_cls with fstype cgroup +device cgroup mounted on /cgroup/blkio with fstype cgroup +device sunrpc mounted on /var/lib/nfs/rpc_pipefs with fstype rpc_pipefs +device /etc/auto.misc mounted on /misc with fstype autofs +device -hosts mounted on /net with fstype autofs +device 1.2.3.4:/storage/NFS mounted on /A with fstype nfs statvers=1.1 + opts: rw,vers=3,rsize=32768,wsize=32768,namlen=255,acregmin=60,acregmax=60,acdirmin=60,acdirmax=60,hard,nolock,noacl,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=1.2.3.4,mountvers=3,mountport=49193,mountproto=tcp,local_lock=all + age: 1136770 + caps: caps=0x3fe6,wtmult=512,dtsize=8192,bsize=0,namlen=255 + sec: flavor=1,pseudoflavor=1 + events: 301736 22838 410979 26188427 27525 9140 114420 30785253 5308856 5364858 30784819 79832668 170 64 18194 29294718 0 18279 0 2 785551 0 0 0 0 0 0 + bytes: 204440464584 110857586443 783170354688 296174954496 1134399088816 407107155723 85749323 30784819 + RPC iostats version: 1.0 p/v: 100003/3 (nfs) + xprt: tcp 733 1 1 0 0 96172963 96172963 0 620878754 0 690 196347132 524706275 + per-op statistics + NULL: 0 0 0 0 0 0 0 0 + GETATTR: 100 101 102 103 104 105 106 107 + SETATTR: 200 201 202 203 204 205 206 207 + LOOKUP: 300 301 302 303 304 305 306 307 + ACCESS: 400 401 402 403 404 405 406 407 + READLINK: 500 501 502 503 504 505 506 507 + READ: 600 601 602 603 604 605 606 607 + WRITE: 700 701 702 703 704 705 706 707 + CREATE: 800 801 802 803 804 805 806 807 + MKDIR: 900 901 902 903 904 905 906 907 + SYMLINK: 1000 1001 1002 1003 1004 1005 1006 1007 + MKNOD: 1100 1101 1102 1103 1104 1105 1106 1107 + REMOVE: 1200 1201 1202 1203 1204 1205 1206 1207 + RMDIR: 1300 1301 1302 1303 1304 1305 1306 1307 + RENAME: 1400 1401 1402 1403 1404 1405 1406 1407 + LINK: 1500 1501 1502 1503 1504 1505 1506 1507 + READDIR: 1600 1601 1602 1603 1604 1605 1606 1607 + READDIRPLUS: 1700 1701 1702 1703 1704 1705 1706 1707 + FSSTAT: 1800 1801 1802 1803 1804 1805 1806 1807 + FSINFO: 1900 1901 1902 1903 1904 1905 1906 1907 + PATHCONF: 2000 2001 2002 2003 2004 2005 2006 2007 + COMMIT: 2100 2101 2102 2103 2104 2105 2106 2107 + +device 2.2.2.2:/nfsdata/ mounted on /B with fstype nfs4 statvers=1.1 + opts: rw,vers=4,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60, acdirmin=30,acdirmax=60,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys, clientaddr=3.3.3.3,minorversion=0,local_lock=none + age: 19 + caps: caps=0xfff7,wtmult=512,dtsize=32768,bsize=0,namlen=255 + nfsv4: bm0=0xfdffafff,bm1=0xf9be3e,acl=0x0 + sec: flavor=1,pseudoflavor=1 + events: 0 168232 0 0 0 10095 217808 0 2 9797 0 9739 0 0 19739 19739 0 19739 0 0 0 0 0 0 0 0 0 + bytes: 1612840960 0 0 0 627536112 0 158076 0 + RPC iostats version: 1.0 p/v: 100003/4 (nfs) + xprt: tcp 737 0 1 0 0 69698 69697 0 81817 0 2 1082 12119 + per-op statistics + NULL: 0 0 0 0 0 0 0 0 + READ: 9797 9797 0 1000 2000 71 7953 8200 + WRITE: 0 0 0 0 0 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 + OPEN: 19740 19740 0 4737600 7343280 505 3449 4172 + OPEN_CONFIRM: 10211 10211 0 1552072 694348 74 836 1008 + OPEN_NOATTR: 0 0 0 0 0 0 0 0 + OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0 + CLOSE: 19739 19739 0 3316152 2605548 334 3045 3620 + SETATTR: 0 0 0 0 0 0 0 0 + FSINFO: 1 1 0 132 108 0 0 0 + RENEW: 0 0 0 0 0 0 0 0 + SETCLIENTID: 0 0 0 0 0 0 0 0 + SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0 + LOCK: 0 0 0 0 0 0 0 0 + LOCKT: 0 0 0 0 0 0 0 0 + LOCKU: 0 0 0 0 0 0 0 0 + ACCESS: 96 96 0 14584 19584 0 8 10 + GETATTR: 1 1 0 132 188 0 0 0 + LOOKUP: 10095 10095 0 1655576 2382420 36 898 1072 + LOOKUP_ROOT: 0 0 0 0 0 0 0 0 + REMOVE: 0 0 0 0 0 0 0 0 + RENAME: 0 0 0 0 0 0 0 0 + LINK: 0 0 0 0 0 0 0 0 + SYMLINK: 0 0 0 0 0 0 0 0 + CREATE: 0 0 0 0 0 0 0 0 + PATHCONF: 1 1 0 128 72 0 0 0 + STATFS: 0 0 0 0 0 0 0 0 + READLINK: 0 0 0 0 0 0 0 0 + READDIR: 0 0 0 0 0 0 0 0 + SERVER_CAPS: 2 2 0 256 176 0 0 0 + DELEGRETURN: 0 0 0 0 0 0 0 0 + GETACL: 0 0 0 0 0 0 0 0 + SETACL: 0 0 0 0 0 0 0 0 + FS_LOCATIONS: 0 0 0 0 0 0 0 0 + RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0 + SECINFO: 0 0 0 0 0 0 0 0 + EXCHANGE_ID: 0 0 0 0 0 0 0 0 + CREATE_SESSION: 0 0 0 0 0 0 0 0 + DESTROY_SESSION: 500 501 502 503 504 505 506 507 + SEQUENCE: 0 0 0 0 0 0 0 0 + GET_LEASE_TIME: 0 0 0 0 0 0 0 0 + RECLAIM_COMPLETE: 0 0 0 0 0 0 0 0 + LAYOUTGET: 0 0 0 0 0 0 0 0 + GETDEVICEINFO: 0 0 0 0 0 0 0 0 + LAYOUTCOMMIT: 0 0 0 0 0 0 0 0 + +device nfsserver1:/vol/export1/bread_recipes mounted on /C with fstype nfs statvers=1.1 + opts: rw,vers=3,rsize=65536,wsize=65536,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=5.4.3.2,mountvers=3,mountport=635,mountproto=udp,local_lock=none + age: 1084700 + caps: caps=0x3fc7,wtmult=512,dtsize=32768,bsize=0,namlen=255 + sec: flavor=1,pseudoflavor=1 + events: 145712 48345501 0 2476 804 1337 49359047 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 + bytes: 0 0 0 0 0 0 0 0 + RPC iostats version: 1.0 p/v: 100003/3 (nfs) + xprt: tcp 871 1 1 0 0 181124336 181124308 28 1971647851 0 1100 807885669 90279840 + per-op statistics + NULL: 1 2 0 44 24 0 0 0 + GETATTR: 145712 145712 0 22994472 16319744 532 107480 109969 + SETATTR: 0 0 0 0 0 0 0 0 + LOOKUP: 2553 2553 0 385932 476148 9 1695 1739 + ACCESS: 596338 596338 0 79281020 71560560 2375 228286 237993 + READLINK: 0 0 0 0 0 0 0 0 + READ: 0 0 0 0 0 0 0 0 + WRITE: 0 0 0 0 0 0 0 0 + CREATE: 0 0 0 0 0 0 0 0 + MKDIR: 0 0 0 0 0 0 0 0 + SYMLINK: 0 0 0 0 0 0 0 0 + MKNOD: 0 0 0 0 0 0 0 0 + REMOVE: 0 0 0 0 0 0 0 0 + RMDIR: 0 0 0 0 0 0 0 0 + RENAME: 0 0 0 0 0 0 0 0 + LINK: 0 0 0 0 0 0 0 0 + READDIR: 0 0 0 0 0 0 0 0 + READDIRPLUS: 0 0 0 0 0 0 0 0 + FSSTAT: 1698 1698 0 250080 285264 6 929 951 + FSINFO: 34 34 0 4352 5576 0 5 5 + PATHCONF: 1 1 0 128 140 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 + +device nfsserver2:/tank/os2warp mounted on /D with fstype nfs4 statvers=1.1 + opts: rw,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=3,acregmax=60,acdirmin=30,acdirmax=60,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.66.88.239,local_lock=none + age: 2 + impl_id: name='',domain='',date='0,0' + caps: caps=0xffbfff7,wtmult=512,dtsize=32768,bsize=0,namlen=255 + nfsv4: bm0=0xfdffafff,bm1=0x40f9be3e,bm2=0x28803,acl=0x0,sessions,pnfs=not configured,lease_time=90,lease_expired=0 + sec: flavor=1,pseudoflavor=1 + events: 1 112 0 0 1 3 117 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 + bytes: 0 0 0 0 0 0 0 0 + RPC iostats version: 1.1 p/v: 100003/4 (nfs) + xprt: tcp 763 0 2 0 2 39 39 0 42 0 2 0 3 + per-op statistics + NULL: 1 1 0 44 24 0 0 1 0 + READ: 0 0 0 0 0 0 0 0 0 + WRITE: 0 0 0 0 0 0 0 0 0 + COMMIT: 0 0 0 0 0 0 0 0 0 + OPEN: 0 0 0 0 0 0 0 0 0 + OPEN_CONFIRM: 0 0 0 0 0 0 0 0 0 + OPEN_NOATTR: 0 0 0 0 0 0 0 0 0 + OPEN_DOWNGRADE: 0 0 0 0 0 0 0 0 0 + CLOSE: 0 0 0 0 0 0 0 0 0 + SETATTR: 0 0 0 0 0 0 0 0 0 + FSINFO: 1 1 0 168 164 0 0 0 0 + RENEW: 0 0 0 0 0 0 0 0 0 + SETCLIENTID: 0 0 0 0 0 0 0 0 0 + SETCLIENTID_CONFIRM: 0 0 0 0 0 0 0 0 0 + LOCK: 0 0 0 0 0 0 0 0 0 + LOCKT: 0 0 0 0 0 0 0 0 0 + LOCKU: 0 0 0 0 0 0 0 0 0 + ACCESS: 3 3 0 600 504 0 1 1 0 + GETATTR: 2 2 0 364 480 0 1 1 0 + LOOKUP: 3 3 0 628 484 0 1 1 2 + LOOKUP_ROOT: 0 0 0 0 0 0 0 0 0 + REMOVE: 0 0 0 0 0 0 0 0 0 + RENAME: 0 0 0 0 0 0 0 0 0 + LINK: 0 0 0 0 0 0 0 0 0 + SYMLINK: 0 0 0 0 0 0 0 0 0 + CREATE: 0 0 0 0 0 0 0 0 0 + PATHCONF: 1 1 0 160 116 0 0 0 0 + STATFS: 1 1 0 164 160 0 0 0 0 + READLINK: 0 0 0 0 0 0 0 0 0 + READDIR: 1 1 0 224 11968 0 1 1 0 + SERVER_CAPS: 2 2 0 336 328 0 1 1 0 + DELEGRETURN: 0 0 0 0 0 0 0 0 0 + GETACL: 0 0 0 0 0 0 0 0 0 + SETACL: 0 0 0 0 0 0 0 0 0 + FS_LOCATIONS: 0 0 0 0 0 0 0 0 0 + RELEASE_LOCKOWNER: 0 0 0 0 0 0 0 0 0 + SECINFO: 0 0 0 0 0 0 0 0 0 + FSID_PRESENT: 0 0 0 0 0 0 0 0 0 + EXCHANGE_ID: 2 2 0 480 200 0 2 2 0 + CREATE_SESSION: 1 1 0 200 124 0 0 0 0 + DESTROY_SESSION: 0 0 0 0 0 0 0 0 0 + SEQUENCE: 0 0 0 0 0 0 0 0 0 + GET_LEASE_TIME: 0 0 0 0 0 0 0 0 0 + RECLAIM_COMPLETE: 1 1 0 128 88 0 107 107 0 + LAYOUTGET: 0 0 0 0 0 0 0 0 0 + GETDEVICEINFO: 0 0 0 0 0 0 0 0 0 + LAYOUTCOMMIT: 0 0 0 0 0 0 0 0 0 + LAYOUTRETURN: 0 0 0 0 0 0 0 0 0 + SECINFO_NO_NAME: 0 0 0 0 0 0 0 0 0 + TEST_STATEID: 0 0 0 0 0 0 0 0 0 + FREE_STATEID: 0 0 0 0 0 0 0 0 0 + GETDEVICELIST: 0 0 0 0 0 0 0 0 0 + BIND_CONN_TO_SESSION: 0 0 0 0 0 0 0 0 0 + DESTROY_CLIENTID: 0 0 0 0 0 0 0 0 0 + SEEK: 0 0 0 0 0 0 0 0 0 + ALLOCATE: 0 0 0 0 0 0 0 0 0 + DEALLOCATE: 0 0 0 0 0 0 0 0 0 + LAYOUTSTATS: 0 0 0 0 0 0 0 0 0 + CLONE: 0 0 0 0 0 0 0 0 0 + COPY: 0 0 0 0 0 0 0 0 0 + OFFLOAD_CANCEL: 0 0 0 0 0 0 0 0 0 + LOOKUPP: 0 0 0 0 0 0 0 0 0 + LAYOUTERROR: 0 0 0 0 0 0 0 0 0 + COPY_NOTIFY: 0 0 0 0 0 0 0 0 0 + GETXATTR: 0 0 0 0 0 0 0 0 0 + SETXATTR: 0 0 0 0 0 0 0 0 0 + LISTXATTRS: 0 0 0 0 0 0 0 0 0 + REMOVEXATTR: 0 0 0 0 0 0 0 0 0 + LAYOUTRETURN: 0 0 0 0 0 0 0 0