The primary LDMS store plugins support:
Librabbitmq 0.8 based, no changes planned. Tracking of librabbitmq updates expected.
- Remove hard-coded limit on number of instances.
- Extended flush options to manage latency and debugging issues.
Ideas under discussion:
- Timestamping of set arrival at the store. - under consideration
- Possible inclusion of final agg time in the writeout
- Duplicate set instance detection (the same set instance arriving by distinct aggregation paths is silently stored twice).
- Conflicting schema detection (set instances with schema of same name and different content, result storage loss or silent error).
- Handle new schema delete old schema. Need this for dvs and perhaps more papi. Handle N schema.
- Check handling start/stop/load/unload. Multiple instance support?
- File permissions and naming
- File owner/permissions set at create has been added to 3.4.7 and 4.x.
- Also want YYYYMMDD naming convention instead of epoch.
- Users want this ability at the start of the file, not just at close/rename.
- Rollover at subday intervals - think option 1 is sufficient for now. Also fixed name would be in alignment with production system usage, so should be considered.
- This could be done instantly just using rollover option 1 with an interval less than 86400 seconds. This would drift unless we add some interval/offset semantics (but in minutes).
- LDMS core managed state pointers (void *) per client(transform policy/store policy)
- Lack of these is making the store and transform APIs very difficult to finish.
- The plugin can hang off that void* (udata of the right flavor), any thing it needs.
SOS is in rapid development, and the corresponding store is tracking it.
Production use of the flatfile store has led to a number of requested changes (below). These changes are sufficiently complicated that an alternately named store (store_var) is in development. The flatfile store will remain unchanged, so that existing production script use can continue per site until admins have time to switch.
- Flush controls to manage latency.
- Output only on change of metric.
- Optionally with heartbeat metric output on specified long interval.
- Output only of specific metrics.
- Excluding output of specific metrics.
- including producername, job id and component id, for single-job, and single-node use-cases.
- Output of rate, delta, or integral delta values.
- Periodic output at frequency lower than arrival, optionally with selectable statistics on suppressed data.
- Statistics: min, max, avg, miss-count, nonzero-count, min-nonzero, sum, time-weighted sum, dt
- Metric name aliasing.
- Rounded to nearest second time stamps (when requested by the user, who is also using long intervals).
- Check and log message (once) if a rail limit is observed.
- Rename file after close/rollover following a template string.
- Generation of splunk input schema.
- Handling of array metrics.