Craig Small [Tue, 28 Apr 2020 09:40:07 +0000 (19:40 +1000)]
build-sys: Enable testing of sigqueue
The referenced commits enavled both pkill and kill to send an integer to
the killed or signalled process. The test_process now will report on the
integer if sent and the testsuite changes take advantage of this
new feature.
Another process make/destroy set had to be made as using spawn
instead of exec changes both the SID and TTY for the underlying
process, making other tests fail.
Craig Small [Fri, 24 Apr 2020 07:42:58 +0000 (17:42 +1000)]
free: Adjust space to really use 9 chars
@steffhip found that while the translation hint said use 9 characters in
the free headers, it really was only 7.
Currently each line is constructed with the following (in non wide format):
Header + 6 Columns. The header takes 7 characters and each column is 11
characters wide and prefixed with one space. Thus we have
7 + (1 + 11) * 6 = 79 characters for each line
By dropping the leading space for the first column after the header -the
header is already terminated by a colon- one could indeed provide the needed
9 letters for the header and thus have 9 + 11 * 1 + (1 + 11) * 5 = 80 Chars
per line which would fit into one line.
Jim Warner [Tue, 14 Apr 2020 05:00:00 +0000 (00:00 -0500)]
library: adapted to the latest lxc conventions (again)
Well, shit! With release 4.0 on March 25th the lxc/lxd
folks have stuck it to us once again. They changed the
cgroup lxc prefix used to identify the container name.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jan Rybar [Tue, 7 Apr 2020 11:41:48 +0000 (13:41 +0200)]
pidof: show worker threads
Reimplementation of pidof for procps toolset contains sort of deactivated code and does not return results for processes without task.cmdline entry (usually kernel worker threads). Old pidof and pgrep do that in comparison. Despite all perks provided by using pgrep instead, pidof should show those workers again.
Jim Warner [Sun, 29 Mar 2020 05:00:00 +0000 (00:00 -0500)]
top: for symmetry with 'E' add 'e' command line switch
Several years after the 'e' & 'E' interactive commands
were introduce to affect memory scaling, an 'E' switch
was added. This was after discovering a dropped Redhat
patch which provided a unique 'M' command line switch.
If only for symmetry it makes sense to offer a similar
command switch ('e') for the Task Area memory scaling.
As was true with 'E', top's help text will show 'e' as
if it were a switch without arguments in order to keep
help text displayed without wrap in an 80x24 terminal.
The man page, however, will show all of the arguments.
The Debian bug referenced below has nothing to do with
locales. In fact, top was made locale independent back
in release 3.3.13 (April, 2018). However, that bug did
reveal some misplaced logic which this patch corrects.
Prompted by the Qualys audit, all rcfile field strings
were checked for potential duplicates which could only
have resulted from some user's manual/malicious edits.
Unfortunately, that code was executed before top had a
chance to enforce the proper/maximum string length (in
the event an extremely old rcfile had just been read).
This created some potential string overrun references.
In top's original 3.3.15 implementation, the potential
overrun extended for 15 characters. That is the number
of field characters added with 3.3.9 (December, 2013).
But, since strchr() was used, no error exit was taken.
In the revised 3.3.16 implementation, the strchr() was
replaced with '&w->rc.fieldscur[n]'. This held overrun
to a single position while producing an error message.
So, this commit just moves that logic to a point where
fieldscur is guaranteed to be longer than EU_MAXPFLGS.
Jim Warner [Wed, 22 Jan 2020 06:00:00 +0000 (00:00 -0600)]
top: restore one line of code to sig_endpgm() function
When that potential abend at program end was addressed
in the patch shown below, one line of code was removed
in error. That line served to suppress some end-of-job
reports should ATEOJ_RPTSTD or ATEOJ_RPTHSH be active.
So, this patch restores that previously deleted logic.
Jim Warner [Sat, 4 Jan 2020 06:00:00 +0000 (00:00 -0600)]
ps: for abnormal end allow core dumps (fix qualys bug)
A Qualys audit patch, represented in the commit below,
added the _exit() call to our abnormal signal handler.
Unfortunately, that disabled the associated core dump.
This patch restores expected behavior of those signals
whose default produces a core dump file + termination.
Jim Warner [Sat, 4 Jan 2020 06:00:00 +0000 (00:00 -0600)]
top: at abnormal end allow core dumps (fix qualys bug)
A Qualys audit patch, represented in the commit below,
added the _exit() call to our abnormal signal handler.
Unfortunately, that disabled the associated core dump.
This patch restores expected behavior of those signals
whose default produces a core dump file + termination.
Jim Warner [Fri, 13 Dec 2019 05:00:00 +0000 (00:00 -0500)]
library: allow negative system DELTA stats, <STAT> api
All TIC delta fields are checked for possible negative
results and set to zero when found. This is done so as
to protect against potential anomalies which depend on
kernel version and/or toggling cpus offline or online.
[ it's probably unnecessary with the latest kernels, ]
[ except for iowait. documentation suggests it might ]
[ decrease which would then create a negative delta. ]
The same approach is employed for most of the 'system'
deltas (ctxt, intr & procs_created). However, with two
of the fields (procs_blocked & procs_running) negative
results were allowed. But it now seems such a division
is unwise so this patch will allow all to go negative.
[ rather than force any 'system' delta value to show ]
[ what's logical, we'll now let all reflect reality. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 11 Dec 2019 05:00:00 +0000 (00:00 -0500)]
top: reposition some data due to a translation comment
In a translator hint, under a caution, a reference was
made to the "next three items". Unfortunately however,
there was one intervening 'item' to which that caution
did not apply. This commit just relocates that 'item'.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Craig Small [Sun, 5 Jan 2020 04:19:41 +0000 (15:19 +1100)]
pgrep: check sanity of SC_ARG_MAX
A kernel change means we cannot trust what sysconf(SC_ARG_MAX)
returns. We clamp it so its more than 4096 and less than 128*1024
which is what findutils does.
Jim Warner [Sun, 13 Oct 2019 05:00:00 +0000 (00:00 -0500)]
top: update copyright notations & fix a man page error
Beyond the copyrights, the single oops in the man page
was introduced in the commit which is referenced below
dealing with some cleanup following that Qualys audit.
This patch will eliminate a bug which is unique to our
newlib branch. It's extremely rare and only happens if
a search ('L'/'&') is initiated during the period when
fields are currently being auto-sized (AUTOX_MODE on).
This bug surfaces as either all zero results for tasks
displayed or a segmentation fault, depending upon what
fields were activated. It is caused by the timing of a
call to the <pids> 'reset' function. When called after
a task refresh, but before do_key(), this bug appears.
So this patch just ensures that 'reset' will be called
after do_key() & before the tasks have been refreshed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 30 Sep 2019 05:00:00 +0000 (00:00 -0500)]
library: removed that 'calloc/free' cruft where doable
Under newlib, the only caller of the readproc routines
is that pids module. And in every case, the address of
some static proc_t structure has always been provided.
As a result, there is no need for the logic supporting
calloc() for a possible NULL pointer which was present
in both of those readproc() and readeither() routines.
Additionally, that pids module takes ownership of most
dynamically acquired 'str' plus 'strv' memory whenever
assigning to a results structure. So, henceforth under
the free_acquired() guy we will only free those string
fields which might exist when not explicitly selected.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 29 Sep 2019 05:00:00 +0000 (00:00 -0500)]
library: removed that 'simple_readtask' unused pointer
This patch just eliminates a parameter present for the
simple_readtask() function which is not needed nor has
it ever actually been used. It will make calls to that
function (via taskreader ptr) slightly more efficient.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 28 Sep 2019 05:00:00 +0000 (00:00 -0500)]
library: removed that 'did_fake' flag from the PROCTAB
When some cleanup was performed on the readproc module
in the commit shown below, some residual code involved
with the 'did_fake' flag remained. Since such logic no
longer served any real need, this patch will whack it.
Jim Warner [Fri, 27 Sep 2019 05:00:00 +0000 (00:00 -0500)]
library: removed that 'SIGNAL_STRING' conditional code
No libprocps user expects signal values to be returned
as 'long long' quantities. More importantly the <PIDS>
api only returns a 'str' result for signal categories.
So this patch eliminates all the conditional code that
depends on the absence of the #define 'SIGNAL_STRING'.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 26 Sep 2019 05:00:00 +0000 (00:00 -0500)]
library: correct one reference typo in that <PIDS> api
This patch will just correct an oops introduced in the
commit shown below. Thank goodness both 'str' & 'strv'
occupy the same storage location in that result union.
Jim Warner [Sat, 10 Aug 2019 00:00:00 +0000 (00:00 +0000)]
top: parallel the master branch pids monitoring change
This commit will just parallel a pending master branch
merge request referenced below. Note, however, that it
is not really needed in this newlib branch because the
procps_pids_select() function itself provides a 'zero'
delimiter with every caller's designated list of pids.
[ since it's required under the master branch, we're ]
[ repeating it here just for documentation purposes. ]
Jim Warner [Tue, 6 Aug 2019 05:00:00 +0000 (00:00 -0500)]
top: avoid a potential SEGV during program termination
The backtrace shown in the bug report referenced below
illustrates a 'normal' program termination interrupted
with some signal, ultimately then causing a top crash.
So this commit just rearranges a little code such that
all signals will be blocked during that rather lengthy
end of program processing regardless of how initiated.
[ in that report, ignore the assertion regarding the ]
[ '-n' option. it obviously was not '1' since do_key ]
[ had been called, which otherwise wouldn't be true. ]
[ and when it is '1' the -d option would be ignored. ]
Jim Warner [Sat, 3 Aug 2019 05:00:00 +0000 (00:00 -0500)]
top: tweak logic associated with keyboard translations
This commit most significant change is the elimination
of the kbd_ENTER entry from that tinfo_tab in iokey().
That entry was a useless artifact left from the commit
which is shown below. It makes no sense to 'translate'
a keystroke into something it already was (i.e. '\n').
The remaining changes just reorder those table entries
for a progression consistent with vim keys: h,j,k & l.
Jim Warner [Sat, 29 Jun 2019 05:00:00 +0000 (00:00 -0500)]
top: standardize PgUp/PgDn management within task area
This commit standardizes the behavior of the PgUp/PgDn
keys when on the main top display. With PgDn, the last
process will become the first process. With PgUp, that
first task will now appear as the last task displayed.
[ this also eliminates some quirks that were evident ]
[ when paging at or near the end of the process list ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 24 Jun 2019 05:00:00 +0000 (00:00 -0500)]
top: attempt to provide missing xterm vim keys support
A recent issue (and merge request) reminded me of gaps
in top's alternate 'vim' navigation keys support. Some
xterm emulators do not pass the customary strings when
keys were used with the <Ctrl> and/or <Alt> modifiers.
While it was a known problem, this issue/merge request
prompted research into the root cause. As it turns out
the problem is traceable to an X resource known by the
name 'eightBitInput'. When 'true' (the default), a key
pressed in combination with <Alt> will not be preceded
by the <Esc> character. Rather, a single character was
presented (modified via an 'eightBitMeta' X resource).
The following approaches would eliminate this problem:
. use: ~/.Xresources with 'Xterm*eightBitInput: false'
. build xterm with 'configure --enable-meta-sends-esc'
( apparently used for CentOS, Fedora, openSUSE, etc. )
. enable xterm's menu via 'configure --enable-toolbar'
( so the user can set the 'Meta Sends Escape' option )
Of course, none of the above steps is desirable from a
user's perspective. So, this patch will add additional
entries to the iokey function's tinfo_tab to represent
strings passed when the <Alt> key does not send <Esc>.
[ hopefully they'll be the same across all platforms ]
Lastly, this patch will also eliminate those redundant
<Atl> + '\', '/', '<' & '>' provisions, which now seem
like overkill and suffer from that same 'eightBitMeta'
xterm problem. And we might as well say goodbye to the
4 '<Alt> + arrow key' table entries (which do not seem
to currently work with any emulator which I can find).
[ what in the world was I thinking way back in 2011? ]
Jim Warner [Mon, 3 Jun 2019 05:00:00 +0000 (00:00 -0500)]
library: add a 'CPU_ZEROTICS' equivalent to <STAT> api
This patch just implements an equivalent to the master
branch 'CPU_ZEROTICS' provision. However, the original
impetus for that earlier implementation was ultimately
attributed to a likely kernel anomaly since corrected.
As a result, in this newlib implementation we take the
opposite approach to the default behavior. There is no
adjustment to TIC_SUM_DELTA values if fewer ticks than
expected are recorded, unless the define is activated.
The commit shown below explains why the 'CPU_ZEROTICS'
define was retained in spite of the fix to the kernel.
Reference(s):
. issue referencing CPU_ZEROTICS
https://gitlab.com/procps-ng/procps/issues/132
. master branch CPU_ZEROTICS summary
commit ee3ed4b45edd66c6e0455d3fab08a48e7ea83030
. lengthy thread leading to CPU_ZEROTICS
https://www.freelists.org/post/procps/CStates-handling-new-switch
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 22 May 2019 05:00:00 +0000 (00:00 -0500)]
library: minimize the use of 'cleanup_stacks' routines
Some parts of our newlib implementation are the result
of functions which have been propagated from module to
module. In particular, those 'cleanup_stacks' routines
are all similar & likely originated in the <pids> api.
In that interface there was a need to free dynamically
acquired memory before the result structure was reused
to satisfy subsequent 'get', 'select' or 'reap' calls.
This, in turn, led to a concept of 'dirty' stacks with
the need to call one of two 'cleanup_stack' functions.
None of the remaining interfaces deal with such memory
yet they each had their own 'cleanup_stack' functions.
Those functions were responsible for resetting each of
the result unions to zero, excluding any 'noop' items.
The bottom line is that for all interfaces, repetitive
calls would require iterating through the stack(s) two
separate times: once to 'cleanup' another to 'assign'.
With this commit we will reduce iterations to just the
'assign' routine. A reset to zero will be accomplished
in the 'extra' item set routine (which is the only one
actually requiring any reset). All other items will be
reinitialized automatically by a new current set value
or upon reallocation when an items compliment changes.
In the <pids> interface, any freeing of dynamic memory
could have been accomplished by adding that 'freefunc'
check to the 'assign' function. However, that requires
an Item_table test with every item. Instead, we'll now
satisfy such needs as the very first step in those set
functions responsible for dynamically acquired memory.
[ the <pids> api retains 2 'cleanup_stack' functions ]
[ to accommodate stack(s) 'reset' & to serve 'unref' ]
Lastly, all the 'itemize_stack' functions were tweaked
by eliminating an unnecessary initialization of result
unions. That objective was already accomplished by the
calloc() in a 'stacks_alloc' function or the remaining
'cleanup_stack' routine found in the <pids> interface.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 9 May 2019 05:00:00 +0000 (00:00 -0500)]
library: remove useless code for 2 'stacks_fetch' guys
These changes are an outgrowth of the research/testing
behind the previous commit. There is no commingling of
select/reap stacks in interfaces beyond the <pids> api
since there's no need to support any 'reset' function.
However, those <pids> changes prompted a review of all
interfaces offering that 'stacks_fetch' function, thus
revealing 2 instances of useless logic/wasted efforts.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 2 May 2019 05:00:00 +0000 (00:00 -0500)]
library: tweak support for get/select/reap, <PIDS> api
When the <pids> api was refactored in the commit shown
below, one objective was enabling the simultaneous use
of 'get' & 'select/reap' functions. Unlike other 'get'
functions, this <pids> 'get' acts as an iterator where
successive calls will return successive tasks/threads.
However, that goal wasn't quite met since a stack used
by 'get' was commingled with the 'select/reap' stacks.
Such commingling supported the 'reset' function, again
a provision which was unique to this <pids> interface.
Unfortunately, some poor assumptions in 'stacks_fetch'
produced a SEGV whenever 'reap/select' followed 'get'.
Thus, this patch addresses those issues and guarantees
such commingled stacks (extents) will be accommodated.
Jim Warner [Thu, 25 Apr 2019 05:00:00 +0000 (00:00 -0500)]
library: correct 'vectorized' string logic, <PIDS> api
The commit referenced below addressed (some) anomalies
surrounding 'strv' pointers. However, there remained a
couple quirks involving a potential NULL return value.
Any NULL values returned from the old library readproc
guys would cause no real harm for newlib. But they did
produce the misleading "[ duplicate ENUM_ID ]" result.
The following all represent potential NULL results and
suggest shortcomings in testing of that earlier patch.
. kernel threads do not have cgroup, cmdline & environ
. even if present environ could require root to access
So, this patch reverts a portion of the earlier commit
and ensures when some vectored string is not available
a traditional dash ('-') is the 'strv' returned value.
[ and we'll also correct one typo in the header file ]
Jim Warner [Tue, 12 Mar 2019 06:00:00 +0000 (00:00 -0600)]
library: refactored some header file items and origins
This commit is intended as a refinement of the patches
mentioned below, where origins/sources of newlib items
were added to the header files for user documentation.
However, if those additions are to be truly effective,
along with kernel documentation (where available), the
following prerequisites must also have been satisfied:
. our identifiers closely align with linux field names
. our derived items are documented or self-documenting
Satisfying those prerequisites prompted this patch and
for these changes, kernel sources were emphasized over
available documentation (shame on me, it should always
have been so). And, while some 'new' fields were found
to be conditional, they were included unconditionally.
These changes appear more extensive than they actually
need be since I have attempted to enforce some spacing
conventions. So, I've summarize the significant things
in the sections that follow. For a proper perspective,
use: 'git diff --ignore-space-change' (good as alias).
___________________________________________ <PIDS> api
This api is unique in that there exists many different
file/directory origins subordinate to /proc/<pid>. And
our item identifiers are sometimes coerced so as to be
able to group related or similar enumerators together.
So, users needed more help relating our identifiers to
an actual documented field. Thus, we will now also add
the field names as with 'stat: delayacct_blkio_ticks'.
Each item ending with a '_C' now consistently includes
both the parent's count/time plus waited for children.
That 'RTPRIO' guy was renamed/relocated as PRIORITY_RT
since its original name is an implementation artifact.
___________________________________________ <STAT> api
The only api change was to correct a typo ('dervied').
_________________________________________ <VMSTAT> api
Even ignoring white space, this interface received the
largest number of changes. Mostly, this was because of
deficiencies in the proc(5) documentation. Recall that
this documentation already sorely lacks any substance.
Usually, just kernel releases are noted, not contents.
When compared to kernel source, that proc(5) contained
many non-existent fields and also omitted many others.
________________________________________ <MEMINFO> api
Sadly, with this api many of the changes were simply a
correction of some earlier 'human error' where several
fields where hashed then tracked but never represented
with an item enumerator in this meminfo.h header file.
_______________________________________ <SLABINFO> api
The 'SLABS' (summary) & 'SLABNODE' items were reversed
since the former are derived from the separate caches.
More significantly, those 'SLABNODE' guys were renamed
to 'SLAB' since they concern individual caches and the
concept of 'nodes' is really an implementation detail.
Also, several enumerators were changed to more closely
agree with official slabinfo(5) documentation referred
to in what we're treating as a base document: proc(5).
Lastly, while those 'SLABS' items are solely a product
of our library and not represented in slabinfo(5), the
names attempt to parallel those found as 'SLAB' items.
______________________________________ <DISKSTATS> api
One enumeration identifier was changed so as to better
reflect its relationship to that actual documentation:
'Documentation/iostats.txt', as referenced in proc(5).
Jim Warner [Mon, 11 Mar 2019 05:00:00 +0000 (00:00 -0500)]
library: repair any broken stuff found during refactor
Rather than offer three separate patches, they've been
consolidated in this single commit. All are related in
that they surfaced while preparing a subsequent patch.
------------------------------------------------------
library: correct a broken '#if define', <SLABINFO> api
It was introduced (embarrassingly) in the patch below.
------------------------------------------------------
library: correct a broken 'GET' macro, <DISKSTATS> api
In the patch referenced below, which purported to make
all the 'GET' macros robust, the 'DISKSTATS_GET' macro
was broken. A necessary parameter wasn't passed to the
subsequently invoked function: procps_diskstats_get().
------------------------------------------------------
library: correct a broken 'sort' func, <DISKSTATS> api
In the commit shown below, an attempt to normalize the
errno handling, the sort function inadvertently lost 1
crucial line of code which produces a consistent SEGV.
Jim Warner [Wed, 13 Feb 2019 06:00:00 +0000 (00:00 -0600)]
library: tweak that lxc container support a final time
Since the patch referenced below traded a compile-time
'sizeof' directive for a run-time 'strlen' call, there
is no need to declare lxc patterns as explicit arrays.
We'll also use the actual lxc patterns by omitting the
beginning slashes ('/') for both of those definitions.
And, looking to the future when most/all lxc users are
using the most recent lxc release, we will make things
slightly more efficient by reversing those two pattern
literals so the most recent pattern was checked first.
Of course, such a change only benefits tasks which are
running in a container. For the majority of processes,
both literals will be compared in that 'if' statement,
assuming the 'LXC' field is currently being displayed.
[ plus, a leftover parenthesis pair has been removed ]
sysctl: do not report set key in case `close_stream` fails
As we're using buffered I/O when writing kernel parameters, write errors
may get delayed until we close the `FILE` stream. As we are currently
outputting the key that is to be set disregarding the return value of
`close_stream`, we may end up in a situation where we report error and
success:
procio: fix potential out-of-bounds access when write fails
When writing to procfs via `proc_write` fails, we try to chunk the
buffer into smaller pieces to work around that issue. When searching for
the next location to split the buffer, though, we can underflow the
buffer in case the current offset is smaller than `LINELEN`. Fix the
issue by passing `cookie->offset` instead of `LINELEN` into `memrchr` in
case `cookie->offset` is smaller than `LINELEN`.
This bug can be triggered on musl-based systems, e.g. by executing
As the value is out-of-range, `write` will return an error and set
`errno` to `EINVAL`. As we're only trying to write a smallish buffer
with a length smaller than `LINELEN` and as the buffer does not contain
any newlines, the call
procio: use the user-supplied delimiter to split large input
The `fprocopen` function allows users to specify a delimiter chacter
that is used to split very large input lines into smaller chunks. While
the code checks that the caller did actually supply the delimiter, it is
in fact never used to split the string. Instead, the hardcoded default
character ',' is always used to split the string.
Jim Warner [Sun, 13 Jan 2019 06:00:00 +0000 (00:00 -0600)]
library: improve header file item comments, <STAT> api
This patch just polishes the 'origin' comments for the
<STAT> header file. In particular those derived/unique
items (the 'SUM' guys) will now be properly explained.
[ in order to employ the 'derived from above' phrase ]
[ with their DELTA versions, all SUM items had to be ]
[ relocated (and some renamed). in turn, that had an ]
[ impact on many portions of the .c source file too. ]
Jim Warner [Fri, 11 Jan 2019 06:00:00 +0000 (00:00 -0600)]
library: remove one needless function call, <STAT> api
This small change is an outgrowth of the research into
the bug represented by that merge request shown below.
With the master branch, a real buglet was subsequently
addressed. In this newlib branch, no bug existed since
the <stat> API relies solely on just cpus reflected in
(and parsed from) the kernel's /proc/stat pseudo file.
[ since that procps_stat_new() priming read about to ]
[ be performed will value info->cpus.total, there is ]
[ no need to separately invoke a procps_cpu_count(). ]
Jim Warner [Thu, 10 Jan 2019 06:00:00 +0000 (00:00 -0600)]
library: adapt readproc for the latest lxc conventions
The merge request shown below prompted (thankfully) an
examination of our lxc containers logic in readproc.c.
As it turns out, the lxc folks changed that eyecatcher
used to identify containers within a task cgroup file.
So this patch, with little extra cost, will enable the
libprocps lxc_containers() guy to handle both strings.
[ additionally, I was shocked to find lxc allows the ]
[ eyecatcher to be changed at ./configure time. such ]
[ a provision has always existed. unfortunately, the ]
[ changed value was only available to root, assuming ]
[ one wished to tackle that undocumented liblxc api. ]
Reference(s):
. what prompted lxc support reevaluation
https://gitlab.com/procps-ng/procps/merge_requests/82
. original lxc support introduced
commit 0557504f9cb84987f9d9038755404be017bdb7d1
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 20 Dec 2018 06:00:00 +0000 (00:00 -0600)]
library: add item origin (as comments) to header files
A lack of documentation seems to be the major obstacle
to releasing this new library. So, in an effort to get
the ball rolling again, this patch adds the origins of
each item as a comment to six of the new header files.
However, before reviewing how such changes may benefit
that documentation objective, it seemed appropriate to
first reflect on newlib's background & current status.
___________________________________________ BACKGROUND
Discussions about and work on a new library began back
in July 2012 but quickly died. After a lull of 2 years
those discussions were resumed in August 2014 but soon
died also (and no code survived the gitorious demise).
With those early discussions, the recommended approach
was to encapsulate all of the libprocps data offerings
in individual functions. When it came to extensibility
it was suggested we should rely on symbols versioning.
Unfortunately that approach would have made for a huge
Application Programming Interface virtually impossible
to master or even document. And, runtime call overhead
would have been substantial for ps and especially top.
So, an alternative design was sought but there were no
new suggestions/contributions via freelists or gitlab.
Thus, in spite of a lack of library design experience,
the procps-ng team (Craig & Jim) set out to develop an
alternative API, more concise and with lower overhead.
Reference(s):
. 07/01/2012, begin library design discussion
https://www.freelists.org/post/procps/Old-library-calls
. 08/12/2014, revival of library design discussion
https://www.freelists.org/post/procps/libprocs-redesign
_____________________________________ DESIGN EVOLUTION
Our newlib branch first appeared on June 14, 2015. And
our current API actually represents the 4th generation
during the past 3 years of evolution. First, there was
a basic 'new', 'get' and 'unref' approach, using enums
to minimize the proliferation of 'get' function calls.
Then, in anticipation of other programs like ps, where
multiple fields times multiple processes would greatly
increase the number of 'get' function calls, a concept
of 'chains' was introduced. This became generation #2.
Such 'chains' proved unnecessarily complex so 'stacks'
replaced them. This was considered the 3rd generation,
but too many implementation details were still exposed
requiring those users to 'alloc', 'read', 'fill', etc.
Finally, a 4th generation emerged representing several
refinements to standardize and minimize those exported
functions, thus hiding all implementation details from
the users. Lastly, handling of 'errno' was normalized.
Reference(s):
. 06/14/2015, revival of new API discussion
https://www.freelists.org/post/procps/The-library-API-again
. 06/24/2015, birth of the newlib branch
https://www.freelists.org/post/procps/new-library
. 06/29/2015, 2nd generation introduced 'chains'
https://www.freelists.org/post/procps/new-library,8
. 07/22/2015, 3rd generation introduced 'stacks'
https://www.freelists.org/post/procps/newlib-stacks-vs-chains
. 06/18/2016, 4th generation refinements begin
https://www.freelists.org/post/procps/newlib-generation-35
. 11/10/2017, 4th generation standardized 'errno'
https://www.freelists.org/post/procps/some-more-master-newlib-stuff
_______________________________________ CURRENT DESIGN
Central to this new design is a simple 'result' struct
reflecting an item plus its value (thanks to a union).
As a user option, these item structures can be grouped
into 'stacks', yielding many results with just 1 call.
Such a 'stack' can be seen as a variable length record
whose content/order is determined solely by the users.
Within that 'result' structure, the union has standard
C language types so there is never a doubt how a value
should be used in a printf statement. Given that linux
requires a least a 32-bit platform the only difference
in capacity surrounds 'long' integers. And, where such
types might be used, the 32-bit maximums are adequate.
The items themselves are simply enumerators defined in
the respective header files. A user can name any items
of interest then the library magically provides result
structure(s). The approach was proven to be extensible
without breaking the ABI (in commit referenced below).
The 6 major APIs each provide for the following calls:
. 'new' ---------> always required as the first call .
. 'ref' -------------------------> strictly optional .
. 'unref' --------> optional, if ill-behaved program .
. 'get' --------------------> retrieve a single item .
. 'select' ----------------> retrieve multiple items .
And the 'get' and 'select' functions provide for delta
results representing the difference between successive
get/select calls (or a 'new' then 'get/select' call).
For the <diskstats>, <pids>, <slabinfo> & <stat> APIs,
where results are unpredictable, a 'reap' function can
return multiple result structures for multiple stacks.
The <pids> API differs from others in that those items
of interest must be provided at 'new' or 'reset' time,
a function unique to this API. And the <pids> 'select'
function requires PIDs or UIDs which are to be fetched
which then operates as a subset of 'reap'. Lastly, the
'get' function is an iterator for successive PIDs/TIDs
returning items previously identified via 'new/reset'.
To provide assistance to users during development, the
special header 'proc/xtra-procps-debug.h' is available
to check type usage against library expectations. That
check is activated by including this header explicitly
or via build using: ./configure '-DXTRA_PROCPS_DEBUG'.
Reference(s):
. 08/05/2016, type validation introduced
https://www.freelists.org/post/procps/newlib-types-validation
commit e3270d463de7eebd9f5ae20c85495e3cb5b69a9f
. 08/11/2016, extensibility while preserving ABI example
https://www.freelists.org/post/procps/new-meminfo-fields
commit 09e1886c9e731f8b8c89a55d11f72f53f030b2de
_________________________ INITIAL DOCUMENTATION EFFORT
The initial attempt, referenced below, dealt primarily
with the <pids> interface. Separate man pages for each
exported function were created. Plus there was another
document describing the items, among other miscellany.
Adopting such an approach encounters several problems:
1. In order to use these man pages, users are required
to already know how to use the library. Or alternately
one could randomly search each of them while trying to
ascertain which function call satisfies their need and
what exactly was the proper compliment/order required.
2. While we can explain what all of those <pids> items
represent, that certainly isn't true for all the APIs.
See the gaps in kernel documentation for <meminfo> and
complete lack of documentation with that <vmstat> API.
3. Our documentation effort should take pains to avoid
unnecessary implementation details. Here's an example:
. "The pointer to info will have memory"
. "allocated and a structure created."
Alternatively, the following conveys user requirements
while not offering any internal implementation detail:
. "You must provide the address of a NULL"
. "info structure pointer."
___________________ RECOMMENDED DOCUMENTATION APPROACH
I recommend that the newlib documentation consist of 3
man pages only. The first would cover the 5 major APIs
and their common functions. The second would deal with
the <pids> API exclusively, explaining how it differs.
Any remaining exported libproc functions which are yet
to be included could be represented in a 3rd document.
For these new documents the following are are assumed:
1. Since we will not be able to document all items, we
shouldn't try to document any items. We should instead
rely on proc(5) or Documentation/filesystems/proc.txt.
2. Program development often involves referencing some
header file(s). So, make that an absolute requirement.
3. With the addition of item origins, represented with
this commit, and considering that 'types' were already
present, the header file might be all some users need.
4. And who knows, when a user of our libproc complains
about gaps in their documentation, it might prompt the
kernel folks to correct those long standing omissions.
To summarize, I suggest that we replace that libproc.3
document with a more general one explaining the basics
of accessing this new library and the common calls for
most of the major interfaces. We can then create a new
document (libproc-pids.3?), which explains differences
in using the <PIDS> application programming interface.
A final document (libproc-misc.3?) covers what's left.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 1 Dec 2018 06:00:00 +0000 (00:00 -0600)]
library: update for fields & latest doc, <MEMINFO> api
This patch will bring the <meminfo> API into line with
that proc(5) document. There were several undocumented
fields that were not noted and these two were omitted:
. 'MmapCopy' was conditional on the #define CONFIG_MMU
. 'Quicklists' depends on the #define CONFIG_QUICKLIST
And we're about to get the following new field in 4.20
which will be represented, at least, in that proc.txt:
. 'KReclaimable' will include SReclaimable plus others
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 1 Nov 2018 05:00:00 +0000 (00:00 -0500)]
top: harden management of 'Hide_pid' array allocations
While setting the size of that Hide_pid array to equal
total pids high water mark was probably safe, in truth
there is no real relationship. At some point one could
exceed that HWM if the 'v' toggle was used extensively
and at least 1 of those entries remained non-negative.
This commit simply divorces Hide_tot from the pids HWM
and bases Hide_pid array size on actual run-time need.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 1 Oct 2018 05:00:00 +0000 (00:00 -0500)]
top: enable alternate '+' placement with collapsed pid
Currently, except for tasks that have no parents, when
a process' children are collapsed the '+' indicator is
shown in the first position within that COMMAND field.
This commit simply provides for indenting the '+' char
so it displays next to that program name/command line.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 18 Sep 2018 05:00:00 +0000 (00:00 -0500)]
top: plug a minor hole in the vertical scrolling logic
In that commit referenced below, a few edge cases were
addressed regarding vertical positioning involving any
'hidden' tasks. But, 2 additional edge cases remained.
In a running top, if the user employed 'other filters'
(o/O) or 'user filters' (u/U) proper vertical position
was not ensured. And, while this could be easily fixed
by striking the home/end or up/down arrow keys, it was
very poor etiquette to shift this burden to the users.
So, this patch plugs that gap, automating the process.
Jim Warner [Sat, 1 Sep 2018 05:00:00 +0000 (00:00 -0500)]
top: eliminated the use of that 'procps.h' header file
That prior patch set the stage for eliminating the use
of that 'procps.h' header, while retaining support for
a ./configure -DXTRA_PROCPS_DEBUG' during development.
This commit just eliminates top's use of 'procps.h' in
favor of a separate include for needed newlib headers.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sat, 1 Sep 2018 05:00:00 +0000 (00:00 -0500)]
library: refactor the XTRA_PROCPS_DEBUG implementation
If we ever were to eliminate the procps.h header file,
as discussed in the thread referenced below, then that
would impair the current XTRA_PROCPS_DEBUG provisions.
The only remaining way to verify result types would be
to explicitly include that <proc/xtra-procps-debug.h>.
So, this commit will once again enable the ./configure
provision for defining the -DXTRA_PROCPS_DEBUG option.
Jim Warner [Wed, 22 Aug 2018 05:00:00 +0000 (00:00 -0500)]
top: a tweak to the forest view collapsed code (again)
From the outset, top has tried to provide some minimal
garbage collection in support of forest view collapse.
For example, with every 'v' keystroke, a check is made
of the currently targeted pids. If all were negative,
which means expanded, that Hide_pid array was emptied.
Recently, yet another efficiency was added wherein the
continuing scan for a targeted pid was terminated when
a match was found. But, one more inefficiency existed.
When a task which was subject to collapse under forest
view mode has disappeared (ended), repeatedly scanning
for such a pid with each iteration makes little sense.
So this commit will negate such targeted pids and thus
avoid scanning every current task looking for a match.
Then, if 'v' is ever stuck at some point in the future
there will be a chance to empty that Hide_pid[] array.
[ hopefully this will be a final tweak of the forest ]
[ view collapse stuff, but cross your fingers anyway ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 9 Aug 2018 05:00:00 +0000 (00:00 -0500)]
top: speed up the collapsed children forest view logic
In forest view mode, once a collapsible parent process
and all of its children (if any) have been identified,
there is no longer a need to scan the remaining tasks.
So this patch will just force a new scan for any other
'Hide_pid' entries which might remain to be identified
after a targeted parent has been completely processed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 1 Aug 2018 05:00:00 +0000 (00:00 -0500)]
top: eliminate all of that 'relative enumerator' bloat
The top program has always specified the maximum stack
depth at 'new' time. Then, in those stacks the minimum
number of result structures were used for representing
only fields actually being displayed in the 4 windows.
That, however, complicated all subsequent access since
each field's enumerator then had to be translated into
a relative position when interacting with the library.
This was accomplished by that Fieldstab 'erel' member.
So this patch eliminates an extra level of indirection
by fully exploiting the existing maximum sized stacks.
Now, the enumerators that top uses to represent fields
also represent their relative positions in each stack.
[ for fields not actually displayed, the position in ]
[ a stack is represented by the 'PIDS_extra' struct. ]
[ thus, there isn't any real library costs for those ]
[ enumerators/fields which aren't currently visible. ]
Signed-off-by: Jim Warner <james.warner@comcast.net>
Craig Small [Wed, 8 Aug 2018 10:13:58 +0000 (20:13 +1000)]
docs: Update ps.1 to warn about command name length
Previous versions of ps used to only match on the first 15 characters
because that's what the kernel used to provide. Newer kernels have a
longer length for this field so procps has been updated to suit.
Jim Warner [Wed, 25 Jul 2018 05:00:00 +0000 (00:00 -0500)]
top: existing 'Inspect' pipe feature now more flexible
Currently, it isn't possible to establish an 'Inspect'
pipe that relies on SIGINT to end. That's because this
signal will also end the parent process (top) as well.
So this patch will temporarily ignore that signal when
processing any 'Inspect' pipe, allowing one like this:
Upon startup there exists the potential for some minor
memory leakage should some rcfile 'Inspect' entries be
invalid. By delaying any malloc/strdup until after the
entries are completely validated we will prevent that.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 18 Jul 2018 05:00:00 +0000 (00:00 -0500)]
top: ensure collapsed children cpu reported accurately
Parent tasks with collapsed children should have their
cpu reflect any unseen tasks only under the following:
1) When built without TREE_VCPUOFF having been defined
2) Exclusively when 'Show_FOREST' display mode was set
3) And only under the current window when in alternate
display mode (except if TREE_VWINALL has been defined)
So, this commit just ensures these objectives are met.
Reference(s):
. issue that began odyssey
https://gitlab.com/procps-ng/procps/issues/99
. original cpu implementation
commit 7c7903e50dff8719408b2a95d75f94ec1407aff7
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 17 Jul 2018 05:00:00 +0000 (00:00 -0500)]
top: make the TREE_VCPUOFF define a tad more inclusive
While this newlib branch did not suffer that SEGV fate
the master branch did with children's %CPU accounting,
it didn't manage the 'TREE_VCPUOFF' #define optimally.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Tue, 10 Jul 2018 05:00:00 +0000 (00:00 -0500)]
top: '^V' is now 'v' (collapse/expand children toggle)
Using Ctrl-V for the collapse children key now appears
as a mistake. First, it's too close to that Ctrl-C key
which would prematurely terminate top. Second, a lower
case 'v' was unused and perfectly compliments an upper
case 'V' which is used to toggle 'forest view' itself.
Jim Warner [Wed, 4 Jul 2018 05:00:00 +0000 (00:00 -0500)]
top: some minor renaming, reformatting and rearranging
This commit just addresses these miscellaneous issues:
. make a few pseudo enum's just a tad more descriptive
. always use 'p' for pointers to a 'struct pids_stack'
. add just a little indent to the 'rSv' macro comments
. rearrange section dealing with new library interface
. always match order of local #undef to parent #define
. forest_begin array index usage is now more efficient
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Mon, 25 Jun 2018 05:00:00 +0000 (00:00 -0500)]
top: parent total cpu includes collapsed children, pgm
Now, when a parent's children have been collapsed, the
cpu used by those unseen tasks will disappear no more.
Instead such tics will be added to the parent's total.
[ if one wished a return to the 'land of lost tics', ]
[ the '#define TREE_VCPUOFF' directive is available. ]
------------------------------------------------------
Note: With collapsible parents now displaying children
cpu usage, it will eventually be noticed the cpu stats
for the summary area and task areas often vary widely.
It's worth a reminder that for top's summary area each
individual cpu and the cpu summary is limited to 100%,
regardless of how many tics a linux kernel may export.
An individual task is limited to 100% times the number
of threads. But, in no case will cpu usage ever exceed
100% times total number of processors. Such limits are
further reduced under 'Solaris' mode ('I' toggle off).
In this mode, a task cpu usage will never exceed 100%.
These limits will now also apply to collapsed parents.
In addition to those influences, results are subjected
to kernel timer sampling anomalies and the distortions
inherent in a small sample size, made worse by smaller
delay intervals. Often there is just 1 or 2 tics for a
few tasks at smaller intervals such as: 1/10th second.
Anyway, should questions on this subject arise, a good
starting point, beyond the reminders above, is the 1st
link listed below. Those other links were derivatives.
Reference(s):
. from the kernel documentation
https://www.kernel.org/doc/Documentation/cpu-load.txt
. as mentioned in the above kernel documentation
https://lkml.org/lkml/2007/2/12/6
. from above, with many more links on the subject
https://www.boblycat.org/~malc/apc/
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Thu, 21 Jun 2018 05:00:00 +0000 (00:00 -0500)]
top: a few tweaks for those scrolling (mostly) changes
This patch just addresses some edge cases with respect
to 'unseen' tasks. Given the ability to preserve other
filters in the rcfile, it's entirely possible the very
first task(s) may not be visible at top startup. Also,
when switching between windows ('a'/'w') we should try
to always position its row #1 on some visible process.
Lastly, a window might have *NO* visible tasks at all.
Therefore, protect 'window_hlp' from an infinite loop.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 20 Jun 2018 05:00:00 +0000 (00:00 -0500)]
top: normalize vertical scrolling for hidden processes
To my knowledge, nobody has ever complained about some
anomalies when scrolling vertically if tasks should be
hidden from view. This can happen with the user filter
('u/U') or other filter ('o/O') features. And although
some tasks are not shown, they still impact scrolling.
This is most apparent when that scroll coordinates msg
is on ('C') & up/down arrow keys used (vs. pgup/pgdn).
Now that we can collapse/expand forked children, there
is a potential for yet more of those hidden processes.
So this commit normalizes vertical scrolling providing
an expected behavior. In other words, the up/down keys
skip the unseen tasks to reposition on a visible task.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 20 Jun 2018 05:00:00 +0000 (00:00 -0500)]
top: refactor the 'scroll coordinates' message support
This patch is simply preparation for upcoming vertical
scrolling enhancements. With those changes, it will be
impossible to predict what the beginning task position
should be at the time the message is currently issued.
This patch will allow such a message to be shown after
the individual windows' tasks have all been displayed.
Signed-off-by: Jim Warner <james.warner@comcast.net>
The issue cited below really dealt with preserving the
'Other filter' criteria in the rcfile. But as an aside
the htop 'F6' feature (collapsed children) was raised.
I took that as an implied challenge and decided to try
implementing a similar feature in top. So, this commit
will now provide a brand new forest view toggle ('^V')
which will be used to collapse/expand forked children.
[ this patch will also lead to additional patches in ]
[ support of more rational vertical scrolling, since ]
[ many more tasks might now be hidden in some window ]
Reference(s):
. where this secondary issue was raised
https://gitlab.com/procps-ng/procps/issues/99
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Wed, 20 Jun 2018 05:00:00 +0000 (00:00 -0500)]
top: to prepare for collapse, move forest view support
In anticipation of a new collapsible child feature, we
will have to make some forest view variables available
to that 'keys_task()' function. This commit just moves
the forest view logic ahead of tertiary input support.
Signed-off-by: Jim Warner <james.warner@comcast.net>
projects / procps-ng / log