Craig Small [Sat, 24 Apr 2021 12:38:48 +0000 (22:38 +1000)]
library: adding IO accounting
This is a modification of MR !122 by @renit1609 to fit the new
library.
Problem statement:
The procps library has no PROC_FILLIO flag to
fetch the proc field "/proc/[pid]/io" data
process-wise.
IO Accounting is not included as part of procps.
Requirement:
We have a requirement to fetch process wise
IO utilization which can be used for monitoring.
When looking through the procps library, I see
that IO Accounting (/proc/[pid]/io) is not being
included as part of procps. There is no such
flag like PROC_FILLIO being included in readproc.h .
Solution:
While looking at the implementation done for
other proc fields, I used the spare bits in app code.
I renamed PROC_SPARE_1 as PROC_FILLIO, the spare bit from
PROC_SPARE_* and used it for fetching /proc/[pid]/io
data as part of the procps library similar to other
fields. I moved the PROC_SPARE_* bits each by 1 bit
to retain the spare bits. Meanwhile added the IO fields
in proc_t structure.
escape.c: Fix missing nl_langinfo on certain configs
nl_langinfo and CODESET are undefined in a musl system. Instead of
uncondionally including langinfo.h, this change includes include/nls.h
which has the tests and work-arounds for systems that don't have these
features. This is similar to how other programs within procps include
langinfo.h via nls.h
Jim Warner [Thu, 22 Apr 2021 05:00:00 +0000 (00:00 -0500)]
library: fix an insidious bug affecting TICS_ALL_DELTA
This simple two line code change fixes an intermittent
bug whereby %CPU for parent(s) with collapsed children
could be vastly understated from those displayed under
the current 3.3.17 publicly available top & libprocps.
If one started several top instances in the background
using very a small delay interval (zero?), then if the
shell under which they were running was collapsed, you
would see similar %CPU results for both the libraries.
However, when running a demanding 'make' like a kernel
compile (especially if backed by fast processors and a
SSD), then newlib would generally show only 1/3 to 1/2
of the collapsed %CPU values that appeared for 3.3.17.
Of course, now that the bug has been swatted with this
commit the disparities between those results is easily
explained. Since newly created tasks never contributed
tics during the interval where they were created, only
with many short lived tasks would differences surface.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 21 Apr 2021 05:00:00 +0000 (00:00 -0500)]
library: lower addr resolution cost for TICS_ALL_DELTA
Rather than run risks of more expensive and repetitive
address resolution, we will establish this local index
for a one time cost and avoid any potential gcc bloat.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 21 Apr 2021 05:00:00 +0000 (00:00 -0500)]
library: lessen chance of distortion on TICS_ALL_DELTA
That old library defined this field as 'unsigned int'.
However, here it was known as a 'signed int'. Thus for
consistency we'll now also treat it as 'unsigned int'.
[ this commit was made in pursuit of a bug involving ]
[ the distortion of elapsed task tics. but, it turns ]
[ out these changes had nothing to do with that bug. ]
[ however, the patch is being retained as desirable. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 31 Mar 2021 05:00:00 +0000 (00:00 -0500)]
top: eliminate the #define VER_J_RCFILE as superfluous
With that commit referenced below, which preserved the
'4' and + '!' toggles in the rcfile, this VER_J_RCFILE
macro was made superfluous. But, it was never removed.
Craig Small [Mon, 5 Apr 2021 04:40:00 +0000 (14:40 +1000)]
pkill: Add lt- variants
The pgrep code checks to see if the program is run as pkill or pidwait
and changes its behaviour accordingly. Some older versions of libtool
run the programs as lt-pkill and lt-pidwait which means the tests fail.
Craig Small [Mon, 29 Mar 2021 11:23:03 +0000 (22:23 +1100)]
build-sys: Handle zero length diskstats in tests
vmstat -d testsuite will fail if your /proc/diskstats is present
but zero length. While this seems buggy behaviour from lxcfs, its
there and its a simple matter to test for it and skip those tests
if we are run on a zero length /proc/diskstats system.
Craig Small [Mon, 29 Mar 2021 11:11:27 +0000 (22:11 +1100)]
docs: psr ps field is last run processor
The ps.1 manpage incorrectly stated that psr field showed the
processor the process was assigned to. However if the assignment
has changed but the process has not run, then the field doesn't
change.
Some digging by @srikard showed it wasn't the processor assigned
but the last one it was run on. The man page now correctly
describes psr in that way.
Craig Small [Mon, 29 Mar 2021 11:07:08 +0000 (22:07 +1100)]
docs: Explain sysctl --system better
The sysctl.8 manpage explained the directory order but not that the
files were then ordered and run in lexiographic order no matter
the directory name.
Shaohua Zhan [Sun, 21 Mar 2021 16:00:00 +0000 (00:00 +0800)]
top: replaced one use of fputs(3) with a write(2) call
This patch is ported from a merge request shown below,
and the following represents the original commit text.
------------------------------------------------------
top: In the bye_bye function, replace fputs with the write interface.
When top calls malloc, if a signal is received, it will
call sig_endpgm to process the signal. In the bye_bye function, if the
-b option is enable, the Batch variable is set, the fputs function
will calls malloc at the same time. The malloc function is not reentrant, so
it will cause the program to crash.
Craig Small [Thu, 11 Mar 2021 11:10:37 +0000 (22:10 +1100)]
slabtop: Check for bad d and o option combination
If you run slabtop with the -d option and then -o option the
delay gets set to zero and it runs forever. slabtop now checks
for this combination and errors.
Adding a DEJAGNU test also found that none of the slabtop
checks were running so they got added to the list and only the
ones that need /proc/slabinfo (if not readable) are skipped.
Jim Warner [Tue, 9 Mar 2021 06:00:00 +0000 (00:00 -0600)]
misc: needed adaptations for the changes in <pids> api
That snowball, which began as a simple removal of some
brackets, now ends with this third patch restoring the
ability to build our project. It was made necessary by
the renaming (and rearranging) of several enumerators.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 9 Mar 2021 06:00:00 +0000 (00:00 -0600)]
library: fix a few source/origin notations, <pids> api
After removing brackets from those 'derived' notations
I was surprised to discover that several origin/source
comments were wrong. So this patch fixes those errors.
[ along the way a couple enumerators were renamed to ]
[ better (i hope) reflect what they're representing. ]
[ that, in turn, also required a little rearranging. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 8 Mar 2021 12:08:21 +0000 (06:08 -0600)]
top: enabled arbitrarily large numbers in 'scale' guys
While experimenting with a new feature, wherein select
fields display the total upon request, the capacity of
the 'num' passed to some 'scale' guys became an issue.
So this commit will, with the compiler's help, put the
responsibility for converting the integer into a float
within the calling code (instead of the called logic).
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 23 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: tweaked key used in hash table, <meminfo> api
This small change was a result of some experimentation
trading our current 'hsearch' hash scheme for 'gperf'.
I discovered that when the ':' character was a part of
each 'gperf' key, that generated search logic was more
complicated and thus slower. But without a ':', it was
a little cleaner/leaner and therefore slightly faster.
Assuming that the same trailing ':' *might* affect the
current 'hsearch' logic, to be safe we will remove it.
[ while the 'gperf' version will slightly outperform ]
[ an 'hsearch', too many ugly implementation details ]
[ were exposed which complicates future maintenance. ]
[ thus, we'll retain our current 'hsearch' approach. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 22 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: normalized the 'read_failed' guys across APIs
This patch will condense some logic in those functions
associated with the file input operations. The changes
will not, for the most part, alter any generated code.
More significantly (though not very) was the change to
two 'strtoul' calls. Since the returned 'endptr' value
isn't exploited, when that parm is set to NULL, we can
save one instruction on each side of such calls (wow).
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 18 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: eliminate a useless conditional in readproc.h
When #define QUICK_THREADS was introduced, for copying
some task data for a child thread, one proc_t pad byte
was used to mark, then later identify, those children.
Later the QUICK_THREADS was recycled as FALSE_THREADS,
and used for a different purpose, but a conditional in
the header file erroneously remained. Now, it is gone!
Craig Small [Sat, 20 Feb 2021 11:30:31 +0000 (22:30 +1100)]
library: Rename to libproc-2
What to call the new library?
Keep using libprocps wouldn't do, its a very different library from
the programs' point of view. It would also mean we could have some
clashes around the packages (two package names, same library name).
The ancient procps used libproc or libproc-a.b.c where a.b.c was the
package version. Kept the revision numbers down (it was always 0.0.0)
but the name of the library changed.
So if we use libproc-2 is there a clash with an ancient procps?
procps v 2.0.0 was around in 1999 so it was 22 years ago, also the
name of the library would have been libproc-2.0.0.so not libproc-2.so
so we're fine with that.
libproc-2 seems to fit, our second major re-work of the procps
library.
Jim Warner [Mon, 15 Feb 2021 06:00:00 +0000 (00:00 -0600)]
docs: expand 2 man pages 'Usage' to include VAL macros
Maybe, the biggest obstacle to successfully exploiting
this new library is after those `stacks' are returned.
Unless a user requests all available `items', there is
always a need to translate an actual enumerator into a
relative position within returned stack(s) of results.
So, this patch attempts to bridge that gap by adding a
brief explanation to the existing discussion in Usage.
[ along the way, 'Usage' & 'Caveats' were refactored ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 14 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: tweak logic for /proc not mounted, <pids> api
Since 'procps_uptime' will access the /proc filesystem
the <pids> 'new' guy should should protect against the
possibility /proc isn't mounted when 'boot_seconds' is
established. A zero is better than the negative value.
[ the only distortion would be to PIDS_TIME_ELAPSED. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 13 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: normalized 'extents_free_all' use across APIs
With the way those 'extents_free_all' guys were coded,
there's no real need to check for a NULL this->extents
before calling 'em. That's how <stat> already does it.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Craig Small [Thu, 28 Jan 2021 11:35:33 +0000 (22:35 +1100)]
build-sys: Ignore temp directory for POTFILES.in
Sometimes the testing sequence would leave a source tree
under procps-ng-(version) which, if update-potfiles was
run, would include these temporary files in the list, causing issues
later.
The script now explicitly ignores those temporary files.
Jim Warner [Fri, 5 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: update those source files for copyright dates
In addition to copyright date, the initial descriptive
line was changed from a generic statement to one which
reflects the specific portion of the proc/ filesystem.
[ such descriptions alternate between 'declarations' ]
[ (h files) & corresponding 'definitions' (c files). ]
Also, a few missing copyright attributions were added.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 2 Feb 2021 06:00:00 +0000 (00:00 -0600)]
library: tweak 'other' user/group names for efficiency
This commit just ensures that the relatively expensive
ID to name conversions aren't performed unless they're
explicitly requested. It also internalizes those flags
that required the PROC_FILLSTATUS flag to also be set.
[ requiring a caller, in our case pids.c, to provide ]
[ two flags when a single field was the objective is ]
[ wrong & represents a future potential toe-stubber. ]
[ moreover, what's worse is that those two flags are ]
[ seemingly unrelated. but, without both, a SEGV can ]
[ can be expected when a result.str pointer is NULL. ]
[ by contrast, in the master branch those fields are ]
[ arrays which, when set to zeroes, produce an empty ]
[ string. So, there is no abend (but no name either) ]
[ when one of those two required flags were omitted. ]
[ and worth noting, in that branch it's not just one ]
[ caller required to observe a two flag requirement. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 20 Jan 2021 06:00:00 +0000 (00:00 -0600)]
docs: adapt procps_misc.3 for new 'misc.h' header file
This commit adapts our man page for a new consolidated
'misc.h' header file. Along the way, some descriptions
were shortened, others lengthened and whitespace added
in an effort to (hopefully) improve readability a bit.
[ the #include subdirectory was also corrected while ]
[ rearranging & grouping functions into 3 categories ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 19 Jan 2021 06:00:00 +0000 (00:00 -0600)]
all: make buildable again for new 'misc.h' header file
With the 4 header files removed in the previous patch,
this commit just changes all those obsolete references
to that new consolidated 'misc.h' header file instead.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 18 Jan 2021 06:00:00 +0000 (00:00 -0600)]
library: combine 4 files into new 'misc.h' header file
Prior to this patch, we had four separate header files
dealing with miscellaneous functions. Those four files
were documented in the single man page: procps_misc.3.
Now, we will have just a single header file documented
in a single man page (that is what's called symmetry).
[ and while we're at it, we will shorten that overly ]
[ long struct 'procps_namespaces' name to agree with ]
[ the function naming conventions, e.g. 'procps_ns'. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 13 Jan 2021 06:00:00 +0000 (00:00 -0600)]
library: reduce overhead for another 'escape' function
The preceding commit made that 'esc_all' function more
efficient by using direct pointer manipulation instead
of an indexed string reference approach within a loop.
This commit applies the same approach to the 'esc_ctl'
function. Now we'll save 12 more iterated instructions
while decreasing the function's code size by 43 bytes.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 12 Jan 2021 06:00:00 +0000 (00:00 -0600)]
library: improve performance for one 'escape' function
While this patch has some cosmetic whitespace changes,
more importantly it makes that 'esc_all' function more
efficient. By abandoning the indexed loop approach for
a direct pointer manipulation, we will save 9 iterated
machine instructions, for a total of 33 bytes of code.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 30 Dec 2020 06:00:00 +0000 (00:00 -0600)]
top: allow setting variable width col scroll (1 vs. 8)
The Inspection feature already offered an INSP_SLIDE_1
provision. This patch now offers similar extensions to
variable width column scrolling (assuming SCROLLVAR_NO
isn't defined). Such a provision was useful during the
development of some recent library UTF-8 enhancements.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 29 Dec 2020 06:00:00 +0000 (00:00 -0600)]
top: relocate the 'fatal_proc_unmounted' function call
With the restoration of corrupt utf8 multibyte editing
to the library, it's important to establish the proper
locale before calling that 'fatal_proc_unmounted' guy.
He calls the original 'look_up_our_self' who, in turn,
will invoke 'stat2proc' which then calls 'escape_str'.
Once there, it's too late to effect changes to locale.
[ the result would be lots and lots of '?' displayed ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 28 Dec 2020 06:00:00 +0000 (00:00 -0600)]
library: refactor 'escape' logic for newlib (2nd time)
Much of what was represented in the commit message for
the reference shown below was revisited in this patch.
It also means that the assertion in the last paragraph
of that message will only now be true with LANG unset.
[ and forget all the bullshit about not altering any ]
[ kernel supplied data. sometimes we must to avoid a ]
[ corrupt display due to a string we can not decode. ]
And while this commit still avoids the overhead of the
'mbrtowc', 'wcwidth' 'isprint, & 'iswprint' functions,
we achieve all the benefits with simple table lookups.
Plus such benefits are extended to additional strings.
For example, both PIDS_EXE and PIDS_CMD fields are now
also subject to being 'escaped'. If a program name did
contain multibyte characters, potential truncation may
corrupt it when it's squeezed into a 15/63 byte array.
Now, all future users of this new library only need to
deal with the disparities between string and printable
lengths. Such strings themselves are always printable.
[ the ps program now contains some unnecessary costs ]
[ with the duplicated former 'escape' functions. But ]
[ we retain that copied escape.c code for posterity. ]
[ besides, in a one-shot guy it's of little concern. ]
Note: Proper display of some multibyte strings was not
possible at the linux console. It would seem a concept
of zero length chars (like a 'combining acute accent')
is not recognized. Thus the display becomes corrupted.
But if utf8 decoding is disabled (via LANG=), then all
callers will now see '?', restoring correct alignment.
Jim Warner [Thu, 24 Dec 2020 06:00:00 +0000 (00:00 -0600)]
library: refactor 'escape' logic for newlib essentials
This new library provides callers with pure strings or
string vectors. It is up to those callers to deal with
potential utf8 multibyte characters and any difference
between strlen and the corresponding printable widths.
So, it makes no sense for the library to go to all the
trouble of invoking those rather expensive 'mbrtowc' &
'wcwidth' functions to ultimately yield total 'cells'.
Thus, this patch will eliminate all the code and parms
that are involved with such possible multibyte issues.
[ Along the way we'll lose the ability to substitute ]
[ '?' for an invalid/unprintable multibyte sequence. ]
[ We will, however, replace ctrl chars with the '?'. ]
[ This presents no problem for that ps program since ]
[ it now duplicates all of the original escape code. ]
[ And, we'll no longer be executing that code twice! ]
[ As for the top program, it takes the position that ]
[ it is wrong to alter kernel supplied data. So with ]
[ potential invalid/unprintable stuff, he'll rely on ]
[ terminal emulators to properly handle such issues! ]
[ Besides, even using a proper multibyte string, not ]
[ all terminals generate the proper printable width. ]
[ This is especially true when it comes to an emoji. ]
[ And should callers chose not to be portable to all ]
[ locales by calling setlocale(LC_ALL, ""), they can ]
[ expect to see lots of "?", regardless of what this ]
[ library fixes in a faulty multibyte string anyway. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 23 Dec 2020 06:00:00 +0000 (00:00 -0600)]
top: added utf8 multibyte support to additional fields
When any process' command line contains multibyte utf8
characters, two separate display problems could arise.
1. If that COMMAND column is not displayed as the very
last field, then field(s) to the right are misaligned.
2. Even when last, should utf8 string length (not that
display length) exceed allowable screen width, it will
nonetheless suffer from improper premature truncation.
Number 1 is less of a concern since the cmdline column
is likely to always be the last field to be displayed,
if only to enable right and left scrolling provisions.
Number 2 is much more likely to occur, especially with
additional fields which might be shown before COMMAND.
Or, forest view child tasks can yield the same effect.
So, this commit will permit the correct utf8 multibyte
display regardless of field position or string length.
And, we'll bring top into line with the ps program for
additional fields potentially subject to utf8 display.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 23 Dec 2020 06:00:00 +0000 (00:00 -0600)]
ps: extend utf8 multibyte support to additional fields
Form its inception (back in May of 2011), escaped_copy
has always been a flawed function. It does not operate
on 'escaped' strings but instead treats all input as a
regular string incapable of containing utf8 sequences.
As such, it should only be used for strings guaranteed
to NOT embody multibyte characters (like SUPGIDS). For
all other strings, which could contain utf8 stuff, the
correct function should have been that escape_str guy.
So this commit changes nearly every escaped_copy call.
Jim Warner [Tue, 22 Dec 2020 06:00:00 +0000 (00:00 -0600)]
library: fix a potential error relating to 'exe' logic
After 'errno' management was standardized, a couple of
fields were added to the <pids> api. Only 1 (PIDS_EXE)
involved dynamic memory and, unfortunately, it did not
conform to that expected normalized ENOMEM convention.
Jim Warner [Sun, 20 Dec 2020 06:00:00 +0000 (00:00 -0600)]
top: add pids_item enumerator as comments to task_show
The Fieldstab uses the full pids_item enumerator names
but also shows top's cryptic relative enumerator names
as comments. So, this commit will mirror that approach
in task_show, adding full pids_item names as comments.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Stephen Brennan [Fri, 20 Nov 2020 00:03:58 +0000 (16:03 -0800)]
Set TZ to avoid repeated stat("/etc/localtime")
With glibc, each time the strftime() function is used (twice per process
in a typical ps -fe run), a stat("/etc/localtime") system call is used
to determine the timezone. Not only does this add extra system call
overhead, but when multiple ps processes are trying to access this
file (or multiple glibc programs using strftime) in parallel, this can
trigger significant lock contention within the OS kernel.
Since ps is not intended to run for long periods of time as a
daemon (during which the system timezone could be altered and PS might
reasonably be expected to adapt its output), there is no benefit to
repeatedly doing this stat(). To stop this behavior, explicitly set the
TZ variable to its default value (:/etc/localtime) whenever it is unset.
glibc will then cache the stat() result.
Craig Small [Mon, 21 Dec 2020 23:55:18 +0000 (10:55 +1100)]
misc: Document sysctl change bug
While sysctl did change the order of /run and /etc to match
systemd in the referenced commit, the Debian bug report that
brought it to light was not documented.
Jim Warner [Mon, 19 Oct 2020 05:00:00 +0000 (00:00 -0500)]
library: a tweak to the UNREF_RPTHASH code, <pids> api
If a hash results report is output (via UNREF_RPTHASH)
a portion is devoted to occupied table entries ordered
by depth. There is a possibility that some depths will
not be found among existing occupied table entries and
to avoid any confusion probably should not be printed.
[ to illustrate the potential for confusion prior to ]
[ this patch, force a very small table size (like 8) ]
[ & then trigger the procps_pids_unref() eoj report. ]
So this patch ensures only 'in use' entries are shown.
[ admittedly, all of the remaining logic in the loop ]
[ could/should be subordinate to this new 'if' test, ]
[ but we will keep the change to a minimum. besides, ]
[ there's no harm subtracting/adding a zero numdepth ]
[ especially since the chance of a zero is very low. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
projects / procps-ng / log