Jim Warner [Thu, 17 Aug 2017 07:22:22 +0000 (02:22 -0500)]
top: protect against the anomalous 'Mem' graph display
Until this patch, top falsely assumed that there would
always be some (small) amount of physical memory after
subtracting 'used' and 'available' from the total. But
as the issue referenced below attests, a sum of 'used'
and 'available' might exceed that total memory amount.
I'm not sure if this is a problem with our calculation
of the 'used' amount, a flaw in the kernel 'available'
algorithms or some other reason I cannot even imagine.
Anyway, this patch protects against such a contingency
through the following single line addition of new code
. if (pct_used + pct_misc > 100.0 || pct_misc < 0) ...
The check for less than zero is not actually necessary
as long as the source numbers remain unsigned. However
should they ever become signed, we'll have protection.
[ Most of the changes in this commit simply separate ]
[ a variable's definition from its associated logic. ]
Jan Rybar [Sat, 5 Aug 2017 22:00:00 +0000 (00:00 +0200)]
top: refresh interval accepts non-locale decimal value
For the past 3 years top has fully honored that locale
LC_NUMERIC setting which impacts his refresh interval.
For the past nearly 5 years top has saved that refresh
value in a locale independent form in his config file.
With this commit we'll intentionally break top so that
a comma or period will be accepted for the radix point
regardless of what that LC_NUMERIC may have suggested.
The current locale LC_NUMERIC will, however, determine
how the delay interval is displayed in the 'd' prompt.
[ This position is better than the approach employed ]
[ by those coreutils 'sleep' and 'timeout' programs. ]
[ Both claim to permit floating point arguments. But ]
[ neither one will accept the comma separator should ]
[ the locale be a country that in fact uses a comma. ]
Jim Warner [Sun, 30 Jul 2017 05:00:00 +0000 (00:00 -0500)]
top: at program startup, avoid a little extra overhead
There was a time when the PROCPS_PIDS_noop represented
the highest valued enumerator. Therefore if one wished
to prime an array consisting of pids_result structures
with this specific item a loop would have been needed.
Now that this enum is the first one, with the value of
zero, we can avoid avoid such a loop with just calloc.
But just in case, we'll use an 'if' to guarantee zero.
[ and the nice thing is, since the value is known at ]
[ compile time, that 'if' test plus subordinate loop ]
[ can be discarded by the compiler as long as it's 0 ]
Jim Warner [Wed, 21 Jun 2017 01:20:20 +0000 (20:20 -0500)]
top: fixing command line parsing errors is now a habit
Ok, I admit it. I'm now tired of cleaning up after me.
This is the 3rd related tweak after that '-1' argument
was originally introduced. And with this patch we will
once again properly honor the '-o' and '-u|U' switches
without a need to be followed by an additional switch.
[ one can follow my unfortunate trail of alterations ]
[ beginning with my most recent fix referenced below ]
Jim Warner [Tue, 13 Jun 2017 05:00:00 +0000 (00:00 -0500)]
library: correct the #define FALSE_THREADS, <PIDS> api
Awhile back, that former QUICK_THREADS #define evolved
into the development (only) FALSE_THREADS which can be
used to ensure a 'duplicate ENUM' convention is output
when certain string fields can't be easily duplicated.
Unfortunately, that original implementation was marred
with zeros being displayed for /proc/$$/meminfo fields
in all the child threads for a multi-threaded process.
So this commit corrects that zero memory field buglet.
Jim Warner [Mon, 29 May 2017 07:22:22 +0000 (02:22 -0500)]
misc: adapt 3 programs to some enum changes, <meminfo>
That preceding commit corrected a little mismanagement
regarding the MEMINFO_MEM_CACHED enumerator, which was
used in the following programs: free, top plus vmstat.
This patch simply adapts those programs to use the new
MEMINFO_MEM_CACHED_ALL enumerator, which reflects both
the 'Cached' plus 'SReclaimable' values they expected.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The <meminfo> module attempted to duplicate the former
sysinfo memory calculations wherein 'SReclaimable' was
added to 'Cached' for the 'kb_main_cached' equivalent.
But, this original approach was wrong for two reasons.
1. The addition occurred too late to impact the 'USED'
calculation which could then cause an underflow in the
top memory display if 'SReclaimable' was heavily used.
2. In changing the actual /proc/meminfo 'Cached' value
it meant that users could not rely on that proc(5) man
page when interpreting the MEMINFO_MEM_CACHED results.
So this commit adds a new enumerator for the inclusive
cached amount plus repositions the calculation so that
a MEMINFO_MEM_USED result will exclude 'SReclaimable'.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 22 May 2017 05:00:00 +0000 (00:00 -0500)]
top: correct functioning of the '-p' command line args
With the introduction of a new '1' command line toggle
I have gone and broken a provision of the '-p' command
line switch (pids monitoring). Multiple pids could not
be specified through the use of comma delimited lists.
Thus, this commit simply corrects that newly added bug
which was born in the 'adjustment' commit shown below.
Werner Fink [Mon, 22 May 2017 12:31:34 +0000 (22:31 +1000)]
sysctl: Print lines longer than 1024 chars
as well do not open /proc/sys files if only the names of the
system control names of the kernel parameters should be shown.
Avoid leaking tmpname in case of a pattern mismatch.
Craig Small [Mon, 22 May 2017 12:15:59 +0000 (22:15 +1000)]
kill: -l space between name parses correctly
This was supposed to be just a cherry-pick of the referenced
commit. However there were two problems:
1. kill code was moved out to its own file
2. strtosig() had a latent bug where signal numbers were not
converted to names.
Original note:
kill -lHUP would work correctly, but kill -l HUP would not.
The list option in kill was hit by a quirk of getopt_long where an
option with an optional argument would not attempt to get the argument
beyond the space, even though a mandatory argument would do that.
The fix is a kludge to scan to the next argument and if it looks
like something we can use, use it. Lucky for us, the list option is
one where parsing can stop immediately.
Jim Warner [Wed, 17 May 2017 05:00:00 +0000 (00:00 -0500)]
top: address the argument parsing quirk involving '-h'
There exists the possibility that a 'putp' call can be
issued before the 'setupterm' invocation has occurred,
as is reflected in a bugzilla report referenced below.
Strangely, such a SEGV isn't always triggered as logic
would suggest it ought to be. I experienced a fault in
these environments with the associated curses version:
. archlinux, procps-ng 3.3.12, ncurses 6.0.20170429
. fedora-25, procps-ng 3.3.10, ncurses 6.0.20160709
. opensuse-42.2, procps-ng 3.3.9, ncurses 5.9.20140201
. gentoo, procps-ng 3.3.12, ncurses 6.0.20150808
. slackw-14.2, procps-ng 3.3.12, ncurses 6.0.20160910
Whereas under these environments there was no problem:
. ubuntu-17.04, procps-ng 3.3.12, ncurses 6.0.20160625
. debian-test, procps-ng 3.3.12, ncurses 6.0.20161126
. mageia-5.1, procps-ng 3.3.9, ncurses 5.9.20140323
[ as an aside, the expected result in the bug report ]
[ is incorrect and should mention the '1' parameter. ]
[ however, until release 3.3.13 when the '1' becomes ]
[ a valid switch, numbers are not detected when used ]
[ with any switch which doesn't require an argument. ]
[ you're welcome to treat that as a separate bugglet ]
Jim Warner [Fri, 12 May 2017 05:03:00 +0000 (00:03 -0500)]
library: set stage for NUMA node field display support
In response to that suggestion referenced below, these
changes allow display of task/thread level NUMA nodes.
Currently, only the 'top' program offers any NUMA type
support and it is limited to the Summary Area display.
With this commit both the 'top' and 'ps' programs will
be able to display NUMA nodes associated with threads.
Jim Warner [Fri, 12 May 2017 05:02:00 +0000 (00:02 -0500)]
misc: eliminate all those remaining gcc -Wall warnings
Reference(s):
proc/readproc.c: In function 'statm2proc'
proc/readproc.c:600:9: warning: variable 'num' set but not used [-Wunused-but-set-variable]
proc/stat.c: In function 'stat_derive_unique':
proc/stat.c:429:1: warning: no return statement in function returning non-void [-Wreturn-type]
ps/parser.c: In function 'arg_type':
ps/parser.c:1098:3: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
ps/parser.c:1099:34: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the 'if'
lib/signals.c: In function 'strtosig':
lib/signals.c:243:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
lib/signals.c:245:13: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the 'if'
slabtop.c: In function 'print_summary':
slabtop.c:223:29: warning: unused variable 'stats' [-Wunused-variable]
watch.c: In function 'process_ansi':
watch.c:232:5: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
watch.c:235:2: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the 'if'
Signed-off-by: Jim Warner <james.warner@comcast.net>
When those standardized 'derived' TIC enumerators were
introduced, a problem with potential DELTA distortions
was also introduced when toggling cpus offline/online.
It has always been true that the 1st (summary) line in
/proc/stat will experience a decrease in total tics if
a new cpu is brought online. Such decreases are mostly
due to reductions in 'idle' and 'iowait' tics. Exactly
why such a counterintuitive phenomenon should occur is
a mystery, but this has been acknowledged in proc.txt.
A separate potential distortion arises with individual
cpus. And, here it extends to both bringing processors
online plus taking them offline too. When that happens
the order of the cpus array tracking is upset, placing
the 'new' values in some other processor's array slot.
But even if we were to occupy the same slot, the issue
regarding reductions in 'idle' & 'iowait' still apply.
In all cases, when a DELTA field was found to be minus
it was forced to zero via the 'TICsetH' macro. However
the 'derived' calculations are subject to new forms of
distortion with their own DELTA values. For example we
could find DELTA_SUM_USER + DELTA_SUM_SYSTEM exceeding
DELTA_SUM_TOTAL, an illogical/inappropriate condition.
So this commit moves former protections for individual
cpus to the stat_derive_unique() function and modifies
it to also extend protections to the 'derived' values.
In the process we now protect the cpu 'summary' counts
which were unfortunately previously overlooked (oops).
Jim Warner [Tue, 21 Mar 2017 05:00:00 +0000 (00:00 -0500)]
top: make command line switch parsing even more robust
This program has always tried to maintain an extermely
robust command line parsing procedure, far more robust
that what's available with the getopt stuff. But, with
the introduction of our first numeric switch it should
have been made even more robust than, in fact, it was.
This commit will now accomplish such a desirable goal.
Jim Warner [Mon, 13 Mar 2017 11:00:00 +0000 (06:00 -0500)]
top: by default, show cmd line vs. cmd name at startup
All of top's display was designed to fit into an 80x24
terminal. This includes the help screens plus both the
Summary and Task Areas, assuming no saved config file.
With release 3.3.10, the startup defaults were changed
assuming ./configure --disable-modern-top wasn't used.
This was done in the hope of introducing some users to
unknown capabilities such as colors, forest view, etc.
The purpose of this commit is to coax a few more users
into possibly exploring another capability: scrolling.
We do so by tweaking the default startup display so as
to show full command lines. Now, when things no longer
fit in 80x24, horizontal scrolling might be exploited.
[ of course, this can be reversed with the -c switch ]
[ this patch has been adapted from the master branch ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 13 Mar 2017 10:00:00 +0000 (05:00 -0500)]
top: provide -1 command line switch as 'Cpu(s)' toggle
If built without ./configure --disable-modern-top, the
program displays each cpu individually providing there
is sufficient vertical screen real estate. For massive
SMP environments this will necessitate use of a config
file where the cpu summary toggle ('1') could be saved
via the 'W' command. But, an rcfile may not be viable.
So this commit introduces a '1' command line switch to
emulate exactly the effects of the interactive toggle.
And since it is our first numeric switch some existing
parsing logic had to be changed slightly. Such changes
are, in truth, an improvement. For example, instead of
seeing "inappropriate '2'" with ./top -2 we'll now see
the vastly more appropriate error "unknown option '2'.
Jim Warner [Mon, 13 Mar 2017 09:00:00 +0000 (04:00 -0500)]
top: provide -E command line switch for memory scaling
In their 3.2.7 version of top, Redhat introduced an -M
switch to automatically scale Summary Area memory data
to avoid truncation (and the resulting '+' indicator).
The procps-ng top does not employ suffixes with memory
data nor does it allow for different scaling with each
separate value. Rather, scaling appears at line start.
If built without ./configure --disable-modern-top, the
Summary Area memory will be scaled at GiB which should
lessen chance of truncation. Otherwise KiB was used to
reflect such memory, increasing the truncation chance.
And while 'W' can be used to preserve some appropriate
scaling value, there are arguments against such rcfile
approaches as cited in the issue and bug report below.
So this commit will bump the Summary Area memory scale
factor from KiB to MiB when using --disable-modern-top
as a concession to that Redhat bug report noted below.
And it also introduces a new command line switch which
can force any desired scaling regardless of the rcfile
or which ./configure option might have been specified.
[ for top's help text we'll show 'E' as if it were a ]
[ switch without arguments in order to keep the help ]
[ text displayable without wrap in an 80x24 terminal ]
[ the man page, however, will show all k-e arguments ]
Jim Warner [Mon, 13 Mar 2017 08:00:00 +0000 (03:00 -0500)]
top: show fewer decimal places for memory (by default)
After much reflection I've come to the conclusion that
displaying 3 decimal places (usually) when memory data
had been scaled is no longer optimal with today's ever
increasing amounts. And given that not all task memory
fields are the same widths, inconsistencies can easily
arise as illustrated and discussed in the issue below.
Instead of unilaterally reducing the number of decimal
places, this commit will sneak in such a change via an
existing configure option that was very likely unused.
The former 'disable-wide-memory' option has now become
'enable-wide-memory', which can be used if the current
behavior (3 decimal places) is preferred. Without that
option, whenever memory is scaled beyond KiB, just one
decimal place will be shown in Summary and Task areas.
And Task area field width will no longer be changed by
this revised configure option. Instead, all such field
widths will now be fixed at the former maximum values.
Craig Small [Sat, 4 Feb 2017 00:44:01 +0000 (11:44 +1100)]
pkill: Return 0 if successfully killed process
Original report:
When trying kill a process with insufficient privileges (see blow),
pkill displays the error message “... failed: Operation not permitted”,
but returns 0. Surely it should return 3?
Return value 0 means one of more things matched. For a pgrep (which
shares code with pkill) this makes sense, there was a match. It seems
wrong for pkill to return 0 when it in fact could not do what you told
it to. However return value 3 means a fatal error and it's not fatal.
Looking at other programs when trying to kill things it cannot kill.
shell kill returns 1, procps kill returns 1, killall returns 1, skill
returns 0 (and says it was successful!, ah well poor old skill)
The consensus seems to be that you return 1 if you cannot kill it, even
if you found it. In other words the return value for both not found and
not able to kill it is the same.
pkill only returns 0 if something was killed. This means we found a
match AND the kill() system call worked too.
Jim Warner [Thu, 26 Jan 2017 06:00:00 +0000 (00:00 -0600)]
top: adapt the master branch XDG specification support
This patch adapts a master branch commit to our newlib
branch. Shown below was that original commit msg text.
------------------------------------------------------
top: Add unobtrusive XDG support
By default the file HOME/.toprc will be prefered. This ensures there
should be minimal breakage even if this file is later created by some
other means. Otherwise we will follow the new behaviour described by
the XDG Base Directory Specification:
If the XDG_CONFIG_HOME environment variable is available we will attempt
to use this as XDG_CONFIG_HOME/procps/toprc otherwise we will fall-back
to HOME/.config/procps/toprc instead.
Jim Warner [Sat, 31 Dec 2016 06:00:00 +0000 (00:00 -0600)]
library <stat>: input file buffer size must be dynamic
Since its introduction, our evolved /proc/stat API has
relied on a static buffer of 8192 bytes. This approach
is probably Ok for other /proc files but it would only
accommodate around 100 processors. If such a threshold
were exceeded then this interface could never succeed.
Now days 100 processors doesn't seem at all excessive.
So this commit trades that static buffer for a dynamic
self-tuning one. And since so much former top CPU code
was already rolled into this module, we just stole the
already proven top dynamic buffer management code too.
[ this also meant switching low level unbuffered I/O ]
[ calls to standard library buffered I/O calls. that ]
[ is exactly what <slabinfo> and <diskstats> employ. ]
Jim Warner [Sat, 31 Dec 2016 06:00:00 +0000 (00:00 -0600)]
library <stat>: improve response to cpu offline/online
With the addition of those new derived SUM values, any
CPUs taken offline or brought online would distort the
historical (delta) results. So this patch just forces
a history reset when such transitions are encountered.
Jim Warner [Sat, 31 Dec 2016 06:00:00 +0000 (00:00 -0600)]
library <slabinfo>: make read function name consistent
For each of those interfaces employing a priming read,
all the other 'read' functions begin with the module's
name except this guy which began with 'read_slabinfo'.
Now, they'll all begin with their module name then end
the same with a '_read_failed' boolean hinting suffix.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 31 Dec 2016 06:00:00 +0000 (00:00 -0600)]
library: eliminate distorted history 1st time switches
Upon reflection, at the point where the 'priming read'
was introduced, any possibility of history distortions
was also eliminated. This was true because all of the
'old' (zeroed) data will have been replaced with 'new'
data whenever a user finally calls get, select & reap.
Thus, any DELTA values will automatically reflect that
interval between 'new' and subsequent retrieval calls.
[ diskstats didn't actually employ a 1st time switch ]
[ like the others so we have changed a comment only. ]
[ but that module will retain something similar used ]
[ inside node_update whenever a new node is created. ]
Jim Warner [Tue, 27 Dec 2016 14:08:08 +0000 (08:08 -0600)]
top: exploits several <stat> new category calculations
This commit just exploits those new library provisions
for tic categories, introduced in the preceding patch,
which had been prompted by the issue referenced below.
[ ok it also corrects the top graph for system usage ]
[ since this turkey failed to include tics for these ]
[ two interrupts: STAT_TIC_IRQ and STAT_TIC_SOFTIRQ. ]
Jim Warner [Tue, 27 Dec 2016 14:08:08 +0000 (08:08 -0600)]
library <stat>: standardized new category calculations
This commit arose out of the discussion (and research)
surrounding the issue cited below. It is an attempt to
consolidate and standardize the calculation of jiffies
categories (e.g. 'idle', 'busy', etc.) once & for all.
Also included is the enum STAT_TIC_NUM_CONTRIBUTORS in
case anyone, in the future, decides to calculate usage
based upon elapsed time * Hz (like top does in process
level %CPU stats). In such an event, a total number of
CPUs or NUMA Nodes would be needed for proper scaling.
Unlike the ps kludge under the master branch to ensure
that namespaces appear the same under both 32 & 64-bit
models, this newlib branch already used a proper type.
However source data still carried the original type as
'signed long' versus that more proper 'unsigned long'.
So, this patch makes sources & destinations identical.
Jim Warner [Sat, 3 Dec 2016 09:03:03 +0000 (03:03 -0600)]
ps: now made responsive to 32 or 64-bit address widths
While a Debian bug report referenced below was limited
to the 'eip' and 'esp' fields, this patch also extends
address width adaptations to some other addresses too.
[ and, we do so in a far less invasive manner than a ]
[ redhat approach shown below adding two new fields! ]
Reference(s):
. new debian bug report
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=846361
. old redhat solution
https://bugzilla.redhat.com/show_bug.cgi?id=244152
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 22 Nov 2016 15:10:10 +0000 (10:10 -0500)]
library: protect against possible 'refcount' underflow
In each module employing a priming read at 'new' time,
should that read fail, a call to 'unref' will be made.
However, there is a hidden dependency that these calls
must never occur before the context 'refcount' was set
due to the way an 'unref' conditional was constructed.
So this commit just ensures that 'unref' will function
as expected, even if called with a 'refcount' of zero.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 11 Oct 2016 05:00:00 +0000 (00:00 -0500)]
top: tweak some stuff relating to non-displayed fields
In the commit referenced below, in addition to several
tweaks to comments, 3 fields were no longer assured of
being present in the results stacks. However, 2 of the
3 fields might, in fact, be required even if they were
not currently being displayed in any of the 4 windows.
The PIDS_CMD is used in two separate 'Inspect' headers
('Y' command) and the PIDS_ID_EUID is required if that
'User Filter' ('u' or 'U' command) was being employed.
That latter field's inclusion will be made conditional
but the former field must be unconditionally included.
( for old top, PIDS_CMD would have always been there )
Jim Warner [Fri, 7 Oct 2016 10:55:55 +0000 (05:55 -0500)]
ps: respond to loss of that PIDS_WCHAN_ADDR enumerator
No longer will ps print nwchan as 'ffffff', '-' or '1'
since the proc/PID/stat wchan field didn't represent a
real address anyway. Rather, the field will henceforth
output a dash ('-'), the ps customary 'not available'.
That man document was also tweaked to better represent
actual behavior. An asterisk ('*') was never shown for
threaded tasks and that dash ('-') usually didn't mean
running tasks (sometimes associated with permissions).
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Fri, 7 Oct 2016 09:44:44 +0000 (04:44 -0500)]
library <stat>: remove that PIDS_WCHAN_ADDR enumerator
Removing the Item_table 'stat' oldflags for WCHAN_ADDR
was wrong since that 'stat' field is not a constant 0.
Rather, it could assume these 3 values: -1, 0, and +1.
I have not been able to pin down a '-1' result, but it
probably means some sort of permission error (-EPERM).
The '1' or '0' values were supposed to distinguish the
tasks that were or were not blocked (whether there was
a wchan address). However, in practice there is little
correlation between those values and availability of a
kernel symbol in /proc/$$/wchan (perhaps due to race).
Anyway, the real point is that a 'stat' wchan does not
now intentionally contain an address. Thus, outputting
'ffffff', '-' or '1' in programs like ps is senseless.
So this patch just eliminates PIDS_WCHAN_ADDR from our
item enumerators leaving only the PIDS_WCHAN_NAME guy.
Now the new library can't be blamed for bad addresses!
Jim Warner [Fri, 7 Oct 2016 07:22:22 +0000 (02:22 -0500)]
top: make that 'forest view' just a tad more efficient
It makes no sense to begin our tracked nested level at
'1' then later require a '1' to be subtracted from the
level as artwork and indentation is added for display.
By beginning such tracked levels at zero, we can avoid
the need to adjust it & use it directly in a snprintf.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 28 Sep 2016 18:13:13 +0000 (13:13 -0500)]
top: adapt for loss of fields obsoleted with linux 2.6
It would have been nice to remove this 'nDRT' guy from
the fields management screen and man document as well.
But, while this version of top could be made to handle
an older rcfile, the reverse would not have been true.
Besides, it's been zero for so long already we'll just
include a 'deprecated' note in top's man page for now.
[ the 'nTH ' field number was corrected in there too ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 28 Sep 2016 16:11:11 +0000 (11:11 -0500)]
library <pids>: remove fields obsoleted with linux 2.6
It seems inappropriate to blindly include fields known
to always be zero in our brand new library. Therefore,
this patch removes support for three such enumerators.
[ that stat 'it_real_value' (PIDS_ALARM) field could ]
[ have been made obsolete before a linux 2.6 release ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 28 Sep 2016 15:10:10 +0000 (10:10 -0500)]
library: strictly cosmetic, absolutely no code changes
This commit just contains some tweaks to comments plus
a few adjustments to whitespace for alignment purposes
and a normalization of the header inclusion #define's.
[ plus a spelling error in one header file was fixed ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 21 Sep 2016 13:08:08 +0000 (08:08 -0500)]
library <stat>: added overlooked numa guest tic counts
When this module was upgraded to 3rd generation in the
patch referenced below, numa node support was migrated
from the top program into newlib. The 'guest_nice' and
'guest' tics were overlooked as top did not need them.
So, this commit corrects that oversight and achieves a
proper symmetry between the cpu & numa jiffies counts.
Jim Warner [Sat, 17 Sep 2016 09:44:44 +0000 (04:44 -0500)]
library: add priming read at 'new' time <most modules>
A priming read at 'new' time in that <slabinfo> module
was important so that permission problems are detected
early. Plus, it also had the potential of making delta
values valid when 'get' or 'select' were first called.
It is for that latter reason that such a read was also
incorporated in the <diskstats> module 'new' function.
No other module, however, employed such priming reads.
This patch just brings those potential benefits to all
of our other newlib modules with the exception of that
<pids> guy. That module is, of necessity, sufficiently
different from those others to justify such exclusion.
Not only are there precious few DELTA enums in <pids>,
but the costs of a priming read would be much greater.
[ otherwise, these newly added priming reads have no ]
[ measurable negative impact on performance/timings. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 17 Sep 2016 08:33:33 +0000 (03:33 -0500)]
library: summary name now more descriptive, <slabinfo>
The <slabinfo> header provides 3 groups of enumerators
with prefixes of SLABINFO, SLABS & SLABNODE. The first
is strictly user oriented & isn't supported internally
by any structure. The other two, however, have structs
associated with 'em but, unfortunately, 1 is misnamed.
The 'struct slabs_node' is associated with 'nodes' and
supports the enumerators with the SLABNODE prefix. But
the 'struct slabs_hist' was associated with 'hist' yet
supports those enumerators with just the SLABS prefix.
We do not care very much what some structure is called
but we do care about an identifier used manipulate it.
This patch will trade the 'hist' identifier associated
with 'struct slabs_hist' for a more congruous 'slabs'.
[ it's awful when the author can't remember what the ]
[ true meaning of an identifier is after creating it ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 17 Sep 2016 07:22:22 +0000 (02:22 -0500)]
library: improve support of dynamic numa nodes, <stat>
If, in fact, numa nodes are dynamic (like that current
total of on-line cpus) the existing logic was lacking.
It included an early return before checking the total.
So, this commit ensures that the nodes total is always
set or updated consistently in only a single function.
There's no need to set it at the time 'new' is called.
[ and since under our existing code this nodes total ]
[ could never possibly have been zero, the erroneous ]
[ test (with the early return) has now been whacked! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 17 Sep 2016 06:11:11 +0000 (01:11 -0500)]
top: remove explicit references to NUMA_DISABLE define
Since our library is responsible for NUMA support, and
since the top program already accommodates the lack of
NUMA data, there's no reason that #define NUMA_DISABLE
need be explicitly referenced in the top source files.
Ergo, this commit just eliminates all such references.
Now, top will rely only on procps_stat_reap() results.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 13 Sep 2016 13:08:08 +0000 (08:08 -0500)]
NEWS: updated with most recent copy from master branch
This just brings the newlib branch NEWS into line with
the current version from our master branch since those
changes have already been incorporated in this branch.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Still unhappy with a minor memory leak associated with
libnuma, I experimented with omitting the dlclose that
was issued at module's end. For some reason which will
remain a mystery, the valgrind leak then went bye-bye.
So this patch just omits one use of dlclose and relies
on whatever kernel magic is at work to free the memory
when each process ends. We kept, however, the original
code (now commented-out) to serve as a future caution.
There remains one potential (but unlikely) dlclose use
near the original dlopen. But there will be no leak as
that 'numa_node_of_cpu' will not yet have been called.
This seems to be the culprit that triggers such leaks.
None of this libnuma shit would likely have come close
to hitting our fan had the numa developers provided us
with 'new' and 'unref' functions like our newlib does.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Craig Small [Sun, 11 Sep 2016 00:40:47 +0000 (10:40 +1000)]
pgrep: add warning that pattern exceeeds 15 chars
Add a warning if you specify a command over 15 characters and don't
use the -f command.
This is a pick of two patches from master: 24fd260 pgrep: Fix off by one error in line check 4a7f9fc pgrep - adds warning that pattern exceeds 15 chars without
kill: Fix free() with bad pointer on SIG-prefixed signal-name
kill -l SIGHUP (or any other signal-name prefixed with "SIG")
would cause free() to be called with a bad pointer instead of
a pointer to what was allocated. Fix this and add test-case.
Jim Warner [Fri, 9 Sep 2016 10:55:55 +0000 (05:55 -0500)]
pmap: fix printing bug associated with the '-x' option
Ever since its introduction, the 'x' (extended format)
option has employed strncmp to parse those smaps keys.
Such an approach worked well as long as those prefixes
were guaranteed to be unique. But, with the 4.3 kernel
a new 'SwapPss' field was added to those within smaps.
That triggered a 2nd match for the 'Swap' logic which,
in turn, resulted in a duplicate output line of zeros.
So this patch just trades strncmp for strcmp, avoiding
potential future problems when /proc/$$/smaps evolves.
Jim Warner [Fri, 9 Sep 2016 09:44:44 +0000 (04:44 -0500)]
library: normalize stack and history allocation naming
Recent profiling and timings have resulted in improved
newlib performance. This patch completes that process.
It just normalizes naming conventions employed for all
allocations involving reaped stacks & history support.
The modules offering a 'reap' function will also offer
the now standardized corresponding STACKS_INCR define.
The modules which provide dynamic history support will
now have a separate #define called NEWOLD_INCR used in
allocations/reallocations. And, while values currently
are set equal to that STACKS_INCR value, in the future
some reason for divorcing those two may be discovered.
----------------------------- for future reference ---
In those modules which contain the STACKS_INCR #define
it is tempting to specify a large value so as to avoid
repeated calls to malloc/realloc. However, in doing so
an extra runtime price will be paid in 'cleanup_stack'
calls with any iterative programs like top or slabtop.
So, with the current values a balance has been sought.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 21 Aug 2016 05:00:00 +0000 (00:00 -0500)]
top: avoid yet more overhead of accessing /proc/status
After discovering those terrible costs associated with
/proc/status vs. /proc/stat, our library changed so as
to favor the latter if a field could be met by either.
Well, low-and-behold, this top program had chosen some
item enumerators that needlessly caused 'status' to be
accessed when 'statm' could be used instead. And while
top's needs require conversion from pages to KiB, that
is still far less costly than that damn 'status' file.
[ this was found when comparing newlib top against a ]
[ 3.2.8 ancient top since the current master top was ]
[ suffering from the exact same 'status' deficiency. ]
[ and, no way was top-3.2.8 going to beat newlib top ]
[ by 50% - we'll allow only a 1-10% occasional loss! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 21 Aug 2016 05:00:00 +0000 (00:00 -0500)]
library: revert one ancient 'escape_str_utf8' deletion
Profiling revealed a large amount of time spent in the
'escape_str_utf8' function (escape.c) with both of our
NLS branches (newlib and master). That same result was
not seen under an ancient top-3.2.8 program & library.
Well, the 3.2.8 result was ultimately explained by the
absence of a 'setlocale', necessary under NLS support.
Thus, when that ancient library tested for locale, all
it got was 'ANSI_...' & assumed 'UTF-8' wasn't active.
But after a hack to that ancient code to place it on a
par with newlib/master, I still found cost differences
that led me to revisit an old change referenced below.
It turns out that 'iswprint' costs far more than would
a call of 'isprint', even with the extra support code.
So this commit just reverts that five year old change.
This patch contains the following miscellaneous stuff:
. The pids_stacks_fetch() routine might call for newly
allocated stacks to be itemized. However, that job was
already tended to by the pids_stacks_alloc() function.
So, this patch just eliminates a redundant invocation.
------------------------------------------------------
. The concept of 'dirty_stacks' has not kept pace with
the evolving stacks implementation. Originally, stacks
were considered dirty only if free() of dynamic memory
was needed before refreshing any single result struct.
Later, with the introduction of the 'extra' item and a
promise to reset it to zero, 'dirty' was much broader.
So, this patch just treats the dirty flg as others do.
------------------------------------------------------
. Lastly, a word or three about performance & timings.
Tuning efforts concentrated on the <pids> API and top.
And unless an oldlib equivalent to the preceding patch
is applied (favoring stat vs. status), newlib top will
often outperform the oldlib version (obviously wrong).
So assuming /proc/stat is preferred in both libraries,
generally speaking, a cpu and elapsed time increase of
1-5% was found for this new stacks oriented interface.
Of course, there's also an increased memory footprint.
There are some occasions, however, when the newlib top
is at a substantial disadvantage. For example if WCHAN
or TTY is displayed, such items will be present in all
newlib reaped stacks (i.e. every process). But old top
would only incur such overhead with displayable tasks.
Thus, oldlib top could outperform newlib by up to 25%,
for example, if only fields requiring NO library flags
were displayed. However, such a scenario is not likely
since only GID, UID, PID, TGID & WCHAN would be shown.
In the usual case, that overhead associated with WCHAN
and/or TTY is overshadowed by other top runtime costs.
All in all a pleasing outcome I deem quite acceptable.
------------------------------------------------------
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 21 Aug 2016 05:00:00 +0000 (00:00 -0500)]
library: prefer /proc/stat before /proc/status, <pids>
Long ago, in a galaxy far away (when top was in charge
of library FILL flgs) /proc/status was to be preferred
over /proc/stat if a field could be satisfied by both.
This was done to avoid costly 64-bit math emulation in
a 32-bit application due to 'unsigned long long' data.
Well it's time to acknowledge the prevalence of 64-bit
platforms. And in such an environment the cost picture
has shifted significantly. It now costs 14 times (wow)
as much to access /proc/status compared to /proc/stat.
In other words, even with '%llu' variables, a sscanf()
call in stat2proc() beats the pants off that home brew
gperf based hashing employed by the status2proc() guy.
In fact, status2proc incurs higher costs than found in
the most expensive aspect of top's forest view option.
Here's a gprof extract to illustrate the costs. It was
produced with an rcfile requiring fields from both the
/proc/stat & /proc/status pseudo files (among others).
There were 5000 iterations in each of 4 separate gprof
runs subsequently merged into 1 gmon.sum for analysis.
Jim Warner [Sun, 21 Aug 2016 05:00:00 +0000 (00:00 -0500)]
top: minimized default fields requested of our library
After doing some profiling and timings, then comparing
newlib top to the existing 3.3.12 version, I concluded
top should avoid stack results unless actually needed.
Not only will stack depth be kept to minimums, but the
new library can save some otherwise wasted pathlength.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 21 Aug 2016 05:00:00 +0000 (00:00 -0500)]
library: most function names now more profile friendly
This patch will begin some refinements associated with
gprof. Initially, functions names have been changed to
help in identifying potential bottlenecks. This effort
also included the obscure set, free and sort routines.
Plus the following additional modifications were made:
. the stacks_alloc prologue was generalized plus added
to a couple of modules where it had not yet propagated
. a couple of the '// end ...' comments were corrected
. some functions have been formally tagged as 'inline'
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 11 Aug 2016 16:11:11 +0000 (11:11 -0500)]
library: expand fields and break an ABI, <MEMINFO> api
The immediately prior commit demonstrated how our APIs
might be expanded in at some point in the future while
maintaining binary compatibility in previous programs.
However, since we've yet to release the 1st version of
our new library, there's no need to violate alphabetic
ordering just yet. So, this patch restores that order.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 11 Aug 2016 15:10:10 +0000 (10:10 -0500)]
library: expand fields yet maintain ABI, <MEMINFO> api
With the 4.8 kernel, 2 new fields will be added to the
meminfo pseudo file. This commit, soon to be replaced,
is intended as an example of how such changes might be
incorporated plus still maintain binary compatibility.
This actually goes further than is strictly necessary,
by retaining meminfo_item ordering for 'set' functions
and the creation of hash table entries. However, there
is only 1 true requirement, that of Item_table entries
which must always agree exactly with item enumerators.
All of the other changes could be done alphabetically.
Ok, so what happens when an old program encounters the
new expanded meminfo items? Well, if it was thoroughly
tested against an old library, it won't even see those
new fields. On the other hand, if it somehow exceeds a
previous MEMINFO_logical_end, then it will just get an
extra result structure or two, with no real harm done.
[ this patch is being replace by the very next patch ]
[ so that our iniitial newlib release can maintain a ]
[ strict alphabetic ordering in all areas initially! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 10 Aug 2016 12:08:09 +0000 (07:08 -0500)]
misc: use 'VAL' macros to benefit from type validation
These 2 programs accessed newlib stacks directly which
meant incorrect result type specifications couldn't be
detected using our new result type validation feature.
And, while the usage was correct, to put each on a par
with all of our other programs, they now rely on those
newlib offered VAL macros for accessing stack results.
[ the ps and top programs retain direct stack access ]
[ when assignment to some result struct is necessary ]
[ PIDS_extra is used by top to store the forest view ]
[ level, while ps uses it for cooked cpu percentages ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
ps/output.c: include <dlfcn.h> only when necessary
dlopen() functionality is only used when SELinux support is enabled, so
<dlfcn.h> only needs to be included when ENABLE_LIBSELINUX is
defined. This fixes the build in configurations where <dlfcn.h> is not
available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Jim Warner [Tue, 9 Aug 2016 10:55:55 +0000 (05:55 -0500)]
library <STAT>: exclude an #include for --disable-numa
When the numa stuff was imported from the top program,
that #include for dlopen() was not made conditional as
it should have been. Well, here it is being corrected.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 8 Aug 2016 16:22:33 +0000 (11:22 -0500)]
library: if the VAL type is wrong still return a value
Rather than return a 0 result for all VAL type errors,
return what would have been without validation active.
This will enable a program like pgrep to still print a
result even though it used some incorrect type member.
With this commit, our VAL macro validations logic will
behave in exactly the same way as the GET validations.
While warning messages may be issued, except for a bad
enumerator, values will always be returned to callers.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 8 Aug 2016 06:23:45 +0000 (01:23 -0500)]
library: strengthen the VAL macro validation functions
One ought not to assume that random memory access will
always succeed or, when it does, that an obviously bad
item enumerator will always be found at that location.
Thus, this patch corrects some really poor assumptions
associated with the 'xtra_procps_debug.h' header file.
[ and it does so in somewhat contorted ways so as to ]
[ avoid several darn gcc -Wnonnull warning messages! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 6 Aug 2016 18:13:13 +0000 (13:13 -0500)]
top: ensure derived VAL macro management is consistent
With this patch, top has ensured that base library VAL
macros are never found inside function bodies. Rather,
they are used solely to support global derived macros.
Program functions remain free to further tailor macros
but they'll now always be based on the top derivative.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 6 Aug 2016 16:11:11 +0000 (11:11 -0500)]
library: ensure that all those 'GET' macros are robust
When users call the native 'get' functions they have a
responsibility to check that the result struct address
was indeed returned. But when using those 'GET' macros
there was no protection for possible NULL dereference.
So this patch will add some protection for a potential
failure of an underlying 'get' function. And should it
occur then those 'GET' macros will just return a zero.
Plus, we'll also mirror that behavior in the debugging
header should the XTRA_PROCPS_DEBUG #define be active.
And, we might as well add a warning when invalid items
are passed to 'GET' macros, just like we do for 'VAL'.
[ lastly, we added the missing opening parens/braces ]
[ to 2 'GET' macros in that xtra-procps-debug.h file ]
[ which went unnoticed until the qa folks caught up. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Fri, 5 Aug 2016 05:00:00 +0000 (00:00 -0500)]
related: fix inconsistencies in result type references
This patch is a response to errors found in those type
references now that our library allows for validation.
In two cases, former assignments to a result structure
could no longer employ that VAL macro if validation is
active. Thus, direct reference to some stack was used.
For the record, those instances were to be found here:
. ps - uses PIDS_extra to store the cooked pcpu values
. top - uses PIDS_extra to store the forest view level
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Fri, 5 Aug 2016 05:00:00 +0000 (00:00 -0500)]
library: provide for validating result type references
During development, we now have a means for validating
that a type referenced in application code matches the
actual type set by the library. The new feature can be
activated through either of the following two methods:
2) an #include <proc/xtra-procps-debug.h> (single pgm)
[ in the future, one could add a formal configure.ac ]
[ provision. but for now a manual approach is safer. ]
Lastly, for any module which provides a sort function,
the handling for both 'noop' & 'extra' enumerators was
made consistent. Now, 'noop' is not sorted and 'extra'
will be sorted as that module's widest supported type.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Fri, 5 Aug 2016 05:00:00 +0000 (00:00 -0500)]
library: expand VAL macros to include the context parm
This patch will set the stage for validating the types
referenced in the result union. For now, the parameter
representing that 'info' structure will remain unused.
[ and while we're at it, let us correct a faulty GET ]
[ macro in the diskstats header. that puppy missed a ]
[ parm which ain't so good if that guy is ever used! ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 31 Jul 2016 05:00:00 +0000 (00:00 -0500)]
related: adapt for changes in result types, <PIDS> api
This patch is the response to changes in <pids> types.
These additional modifications were also incorporated.
. ps -------------------------------------------------
pr_wname was eliminated as it just duplicated pr_wchan
pr_wchan referenced WCHAN_ADDR in error, vs WCHAN_NAME
pr_nwchan referenced WCHAN_NAME, not proper WCHAN_ADDR
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 31 Jul 2016 05:00:00 +0000 (00:00 -0500)]
library: normalize & minimize result types, <PIDS> api
This commit attempts to minimize the variety of types
currently used. Plus, the following were also changed:
. the MEM fields were switched to parallel the VM guys
. PIDS_MEM_VIRT -> PIDS_MEM_VIRT_PGS
. PIDS_MEM_VIRT_KIB -> PIDS_MEM_VIRT
. made NICE 's_int' so that it then parallels PRIORITY
. change RTPRIO & SCHED_CLASS from 'ul_int' to 's_int'
. removed Item_table 'oldflags' for an obsoleted field
. PIDS_WCHAN_ADDR
. added calculations like TICS_ALL_C for the following
. PIDS_TICS_USER_C
. PIDS_TICS_SYSTEM_C
. these three new 'TICS' fields have been incorporated
. PIDS_TICS_BLKIO - jiffies spent in block i/o
. PIDS_TICS_GUEST - jiffies spent as a guest
. PIDS_TICS_GUEST_C - as above, includes dead children
. that PIDS_TICS_DELTA was renamed PIDS_TICS_ALL_DELTA
( so it did not hide between TICS_BLKIO & TICS_GUEST )
( and to make clearer what's included: utime + stime )
. eliminated 'sl_int' entirely from that result struct
[ often, the <pids> module changes necessitated that ]
[ readproc header and source files had to change too ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
projects / procps-ng / log