Jim Warner [Sun, 28 Jun 2015 05:00:00 +0000 (00:00 -0500)]
top: exploit the new library API for memory statistics
This commit represents the pioneering attempt at using
the concept of 'chained' library requests in an effort
to reduce function call overhead. It required exposing
no more implementation details than were already shown
through the individual calls, yet is satisfied in one.
It is just such an approach that will prove invaluable
when it comes time to access individual /proc/##/data.
Programs could 'chain' only those 'results' structures
representing their current view independent of all the
fields any such programs might be prepared to display.
Thus the standard 'read', which wouldn't apply to task
level data very well (or efficiently), can now become
a 'read_chain' whereby the former PROC_FILL flags need
can be satisfied & yield the minimum open/close calls.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 28 Jun 2015 05:00:00 +0000 (00:00 -0500)]
library: add chaining provisions to meminfo and vmstat
If a caller chooses to reduce the overhead of repeated
function calls, this commit provides for acquiring all
the desired information in just a single library call.
Signed-off-by: Jim Warner <james.warner@comcast.net>
This patch fills in some missing fields which have top
dependencies. Additionally, I've tried to mirror those
calculations Jaromir added for release 3.3.10. The one
calculation that remains missing is 'available' memory
for some kernels. For this API, we'll use a fall-back.
Lastly the lxc safeguards which were recently added to
the old procps library were incorporated here as well.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Jim Warner [Sun, 28 Jun 2015 05:00:00 +0000 (00:00 -0500)]
library: tweak formatting style for meminfo and vmstat
This patch mostly just eliminates darn tab characters.
Plus the library function declarations and definitions
have been standardized. Most visibly, the input params
now have all been indented on their own separate line.
Signed-off-by: Jim Warner <james.warner@comcast.net>
Craig Small [Sun, 21 Jun 2015 08:22:28 +0000 (18:22 +1000)]
library: memory and vmstat API changes
Created new set of functions for meminfo related calls. Liked the
format of that better so changed vmstat around so the look similar.
Missed the makefile change for uptime so added it in now.
Craig Small [Sun, 21 Jun 2015 08:20:04 +0000 (18:20 +1000)]
library: uptime API changes
Removed the printf_uptime, binaries can do printf easily enough.
sprint_uptime split into two as there wasn't a lot of common
code
sprint_uptime(): old style uptime line
sprint_uptime_short(): short new style "uptime -p"
Hertz_hack needed this, no sane system uses the code (I think)
so just assume 100 like we do in FreeBSD.
Craig Small [Fri, 19 Jun 2015 21:43:02 +0000 (07:43 +1000)]
library: sysstat and vmstat api changes
Use the standard libc declarations.
For protecting the headers for C++ procps used to have its
own defines, this change makes them use the standard libc ones.
getstat() -> procps_stat_*
vminfo() -> procps_vmstat_*
These two components of the library now use the newer version of
the API with less exposed global variables. The old methods are
there for now.
Craig Small [Wed, 24 Jun 2015 12:27:24 +0000 (22:27 +1000)]
pwdx: Do not link to libprocps
pwdx doesn't actually use any of the libprocps functions but
it is linked because it is the default. This specific LDADD
removes that unrequired linking.
Jim Warner [Sat, 20 Jun 2015 05:00:00 +0000 (00:00 -0500)]
build-sys: tweak to prevent dirty tree with every make
Please let's stop the nls translation insanity. With a
one time push we can eliminate the dirty tree syndrome
which surfaces with every local build. Later, before a
release, the translations can be updated in final form
then pushed just 1 more time to the gitlab repository.
I'm tired of having to always re-issue this request in
order to circumvent the problem and thus prevent a too
broad commit (not to mention some nasty side effects).
Jim Warner [Sat, 20 Jun 2015 05:00:00 +0000 (00:00 -0500)]
top: sacrifice a redundant procps_linux_version() call
With the commit referenced below, the linux version is
no longer available via an external variable. So we'll
eliminate the extra superficial function call employed
at program end as part of a debugging (only) o/p spew.
[ the user will soon be returned to the command line ]
[ & he/she can run their own 'uname -r' if in doubt! ]
Craig Small [Fri, 19 Jun 2015 11:00:46 +0000 (21:00 +1000)]
library: Change linux version
Added function procps_linux_version() which used to be an
exported integer instead. Also changed the method of obtaining
the linux version (more correctly the os release) to use a specific
procfs entry. This works for both Linux and FreeBSD.
Jim Warner [Thu, 18 Jun 2015 05:00:00 +0000 (00:00 -0500)]
ps: exploit the simplified library interface for wchan
This patch was made necessary by those library changes
in support of recently revised/simplified wchan logic.
In addition, this commit eliminates a broken alternate
'namelist' provision which was intended to allow users
to specify a System.map file to be used in translating
addresses into function names. But, the real effect of
the now defunct 'N' and '-n' options was to indirectly
force addresses (not names) to be displayed since such
user named map files could not be successfully parsed.
Besides when the required FRAME_POINTER kconfig option
is absent there is no address to translate and when it
is present /proc/PID/wchan is already translated. Thus
an alternate mapping is unnecessary and inappropriate.
[ we'll forgive POSIX for documenting '-n namelist' ]
Jim Warner [Thu, 18 Jun 2015 05:00:00 +0000 (00:00 -0500)]
library: refactor and rely on modern kernels for wchan
Several Debian based distributions were recently found
to have omitted a kernel configuration option that had
the effect of rendering /proc/#/stat and /proc/#/wchan
useless for providing any 'sleeping in function' info.
That problem also prompted a reevaluation of the whole
approach to wchan matters which had grown increasingly
complex as our library evolved over the last 13 years.
The net result was a decision to rely on /proc/#/wchan
which arrived along with the 2.5 kernel. This then let
us vastly simplify the internal code plus the external
interface which will benefit both the top and ps pgms.
Craig Small [Thu, 18 Jun 2015 12:37:24 +0000 (22:37 +1000)]
library: remove procps_version functions
It doesn't make any sense to have the binary version strings
embedded into the library. The version strings are defined
already either in the Makefile or in include/c.h
Jim Warner [Sat, 13 Jun 2015 05:00:00 +0000 (00:00 -0500)]
library: add support for displaying LXC container name
This commit adds a lxc container name to every proc_t.
If a process is not running in a container, then a '-'
will be provided, making such a field always sortable.
Unlike other proc_t character pointers, lxc containers
will find many duplicate shared values. So rather than
strdup 'em (with a later free required upon reuse), we
try to keep track of those already seen and share that
address among all tasks running within each container.
We rely on the lines in the task's cgroup subdirectory
which may initially seem somewhat unsophisticated. But
the lxc library itself uses a similar approach when it
is called to list active containers. In that case, the
/proc/net/unix directory is parsed for the '/lxc' eye-
catcher, with potential complications from hashed path
and names that are too long (something we don't face).
[ too bad docker abandoned lxc - our commit won't do ]
[ anything for the users of those kind of containers ]
Jim Warner [Sat, 13 Jun 2015 00:00:00 +0000 (00:00 +0000)]
library: address memory aberration with LXC containers
Under a lxc container, the /proc/meminfo 'MemFree' and
'MemAvailable' amounts will be equal, unless memory is
being limited via cgroups in which case 'MemAvailable'
could exceed that for 'MemTotal'. And when a container
has been nested, there exist additional memory quirks.
A program might then display used or available amounts
greater than total memory (assuming unsigned honored),
or negative values (should a signed cast be employed).
This anomaly primarily impacted the top and free pgms.
Thus, two simple sanity checks have been introduced to
avoid any illogical kb_main_available or kb_main_used.
( Busybox top & free also display anomalous although )
( different results when running in a lxc container. )
Craig Small [Sat, 13 Jun 2015 05:13:39 +0000 (15:13 +1000)]
Skip tests where /proc/vmstat unreadable
testsuite would fail if /proc/vmstat was unreadable.
Issue #3 brought up by Mike Frysinger.
test script explicitly checks to see if it is readable and
sets these tests to unsupported if not.
Craig Small [Sat, 13 Jun 2015 05:04:31 +0000 (15:04 +1000)]
Create test process
For the test suite, procps used to use sleep which would just
create a process or two to test the tools against. Some setups
coreutils creates all programs including sleep into one blob which
means a lot of the tests fail, see issue #2
It allows to distinguish the initial NAME (to be translated) with the
latter one (that must not be translated) and thus permits to handle its
translation differently.
Recent commit 9742c74e7c522 ("pgrep: Enable case-insensitive process matching")
caused the "opts" string to overflow the show 32-character space allocated for
it.
Bump it up to 64 bytes, which should be enough even if more options are added.
Tested: Running ./pgrep stopped crashing and `make check` passed.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
Craig Small [Wed, 13 May 2015 11:57:10 +0000 (11:57 +0000)]
Merge branch 'fix_parallel_build' into 'master'
build-sys: use proper dependencies on libproc.la
Use `LDADD` or `*_LDADD` instead of `AM_LDFLAGS` to refer to `libproc.la`.
This is recommended in the automake manual:
http://www.gnu.org/software/automake/manual/html_node/Linking.html
Before this commit, parallel builds may break, as there is no explicit dependency to ensure the library is built before the binaries that try to link to it.
Tested by running `make -jNN` repeatedly for different levels of parallelism to ensure the build works. Also checked that `make check` and `make distcheck` still work as expected. Also made sure that a parallel make invocation works with `make -j distcheck`.
Reported-by: Jim Warner <james.warner@comcast.net> Signed-off-by: Filipe Brandenburger <filbranden@google.com>
See merge request !2
Use LDADD or *_LDADD instead of AM_LDFLAGS to refer to libproc.la.
Otherwise, parallel builds will break as there is no explicit dependency
to ensure the library is built before the binaries that try to link to
it.
v2: Added empty rules lib_test_*_LDADD to remove the dependency on
libproc which is not used by the lib/test_* binaries.
Tested by running `make -jNN` repeatedly for different levels of
parallelism to ensure the build works. Also checked that `make check`
and `make distcheck` still work as expected. Also made sure that a
parallel make invocation works with `make -j distcheck`.
Reported-by: Jim Warner <james.warner@comcast.net> Signed-off-by: Filipe Brandenburger <filbranden@google.com>
William Orr [Sun, 10 May 2015 07:42:09 +0000 (17:42 +1000)]
pgrep: Enable case-insensitive process matching
FreeBSD has case-insensitive matching of processes in pgrep and
pkill, which can be super-useful. This patch uncomments and
documents the code needed to make this work.
Craig Small [Sun, 10 May 2015 07:23:54 +0000 (17:23 +1000)]
docs: Updated documentation
Updated the documents with the following general changes:
* Replaced Gitorious with GitLab
* Moved translate stuff out of README
* Changed plain text to markdown (looks better on website)
tests: slabinfo should not be too strict about slab names
Before this commit, the test checking `vmstat -m` (slabinfo) output uses
a fairly strict regular expression that only allows alphanumeric
characters and a few exceptions such as "_", "-", "(" and ")".
However, recent kernels use a wider range of characters, such as ">".
For instance, see this Linux commit which creates a "page->ptl" slab:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/mm/memory.c?id=b35f1819acd9243a3ff7ad25b1fa8bd6bfe80fb2#n4283
Other patches for reporting slab usage per memcg include the names of
the cgroup in the slabinfo output, which can include additional
characters and use dots for abbreviation.
The check should not be so string, instead it could simply look for a
chain of non-whitespace characters and that should be enough.
Tested that `make check` is still working, including in some of the
environments where features that enable the additional slabinfo names.
Signed-off-by: Filipe Brandenburger <filbranden@google.com> Signed-off-by: Craig Small <csmall@enc.com.au>
tests: fix regexp in ps_sched_batch.exp to match in first line
The current regexp checks for a \s+ in the beginning, however that will
only match if there is a \n in the `ps` output before test-schedbatch,
but that will not happen if test-schedbatch is the first process in the
list, which happens if the PID of test-schedbatch is low enough to bring
it up in the sorted list.
Fix it by enabling newline-sensitive matching with (?n) which then
allows using ^ and $ anchors in the regexp (including an optional \r
introduced by expect.) Matching the end of line also improves checking
that the last field matches 18 exactly and not something like 181, etc.
Tested that `make check` does not break and also fixed the flakiness
seen in an environment with few processes running under the test user
which made the issue more frequent.
Signed-off-by: Filipe Brandenburger <filbranden@google.com> Signed-off-by: Craig Small <csmall@enc.com.au>
Right now the test case is both testing it (expect_pass "$test") and
marking it as untested (untested "$test"), it should do either one or
the other, so stop marking it as untested.
Before this change, these lines appear in testsuite/ps.log or the output
of `make check RUNTESTFLAGS="--all"`:
Craig Small [Sat, 9 May 2015 07:53:08 +0000 (17:53 +1000)]
pgrep SID=1 is valid
On most systems the only process with a SID=1 is init
and certainly not a test sleep. On docker systems this
test program IS on SID=1 and so our "impossible SID" becomes
possible.
build-sys: install binaries in bindir (get rid of usrbin_execdir)
The "usrbin_execdir" hack meant to install some binaries in /bin and
others in /usr/bin. However:
- It is very inflexible: not much control on the final directory name
and it is not possible to get rid of the usr/bin suffix without
patching the build system.
- It is hard to use: it requires configure to receive --exec_prefix=/
and other settings do not make much sense. It is not very obvious that
that setting needs to be passed and it takes a while to figure it out.
- It produces garbage with the default setup: the default prefix of
/usr/local ends up installing the binaries under /usr/local/usr/bin
which does not make any sense.
Furthermore, the requirement to split binaries in /bin and /usr/bin is
not that strong since some distributions adopted the /usr merge and so
would agree to just deploy all binaries to /usr/bin directly.
Distributions that would still like to split /bin from /usr/bin should
actually move binaries such as `ps` and `kill` to /bin after the install
of procps-ng is complete. After all, they are the ones responsible for
determining what are the binaries that need to be in the root partition
and that list depends on their early boot init scripts, so it is
possible that the list must be augmented with other binaries from this
package.
Therefore, I propose here to get rid of that hack and simply install all
the binaries to bindir instead, which solves the problems described
above and simplifies the build and install of procps-ng.
Tested that it builds and both `make check` and `make distcheck` work.
Tested that `make install` works and produces the expected tree, the
only difference being the absence of the bogus /usr/local/usr/bin
directory and now all binaries are merged into /usr/local/bin as
expected.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
Otherwise, automake 1.14 will warn that this option will become the
default in an upcoming release, which will cause problems for the
procps-ng build.
Now that the automake rules were merged in the top level Makefile.am,
it is possible to enable "subdir-objects" without breaking the build or
the dist.
Tested that it builds and both `make check` and `make distcheck` work.
Tested that `make install` works and produces the same tree before and
after this change. Confirmed that binaries are also placed in the same
locations in the build tree.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
This will be required for subdir-objects, otherwise automake will have
problems with more than one Makefile.am having rules to build the same
files.
Tested that it builds and both `make check` and `make distcheck` work.
Tested `make install` and compared the tree with the one installed
before this commit, both installed the binaries to the same locations.
The binaries are also in the same location in the build tree (for
instance, ps/pscommand is still there.)
Checked the binaries for the correct libraries linked into them. Binary
sizes matched before and after this change.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
build-sys: drop unneeded $(top_srcdir) from source paths
This is cleaner and we need to match paths exactly when we enable the
subdir-objects automake option.
Out-of-tree builds still work since automake is smart to know these are
source files and that it needs to look for them in $(top_srcdir), so
there is no need to make this explicit.
Tested that it builds and both `make check` and `make distcheck` work.
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
build-sys: split test cases in lib/ into their own files
In order to avoid compiling the same source files twice, with and
without the TEST_PROGRAM define.
Tested that the build still works and that `make distcheck` works as
expected.
Tested that the test_* programs in lib/ keep working. (Though they are
not really invoked by `make check` and in particular test_nsutils is
quite useless, test_fileutils also quite poor.)
Signed-off-by: Filipe Brandenburger <filbranden@google.com>
Craig Small [Fri, 3 Apr 2015 08:18:58 +0000 (19:18 +1100)]
free: Use IEC units
Free always used 1024 based units but used the confusing old style
kilo,mega etc.
This change changes the names to kibi,mebi for 1024 based divisors
and kilo,mega for 1000 based divisors or IEC units.
It also checks if you try to set two units, e.g free -k -m
Petabyte and Pebibyte have been added.
If you used to use the long options such as --mega these will now
actually print megabytes (they previously printed mebibytes).
The short options are being used on the IEC units
The loop that parses options has a of by one bug where the realloc
adds one byte, instead of one list element. This is exposed when
you try things like:
pgrep -t,,,,
Craig Small [Tue, 17 Mar 2015 11:43:22 +0000 (22:43 +1100)]
pgrep.1 removed STANDARDS section
pgrep got updated due to the confusion of the f,l and a flags.
While the newer behaviour is far better but it is no longer following
the ancient Solaris standards, so that got removed.
Jaromir Capik [Mon, 2 Mar 2015 17:41:07 +0000 (18:41 +0100)]
w: fixing missing '-' in the FROM field when empty
With 99bebff06a058f1ebf794fca1b358d0df73fdd35 a configurable
width of the FROM column was introduced. Unfortunately this
caused a regression in the dash printing. Hopefully fixed
with this commit.
Craig Small [Sat, 24 Jan 2015 07:53:29 +0000 (18:53 +1100)]
pmap: print process even if smaps unreadable
pmap would previously print the process name if
/proc/PID/smaps could be opened, even if subsequent
reads failed. This actually occurs with other users
PIDs.
Kernel 3.18rc1 introduced a change where the file could
not been opened, meaning pmap -X 1 previously showed
the process name and nothing else but NOW shows nothing
make check failed because of this.
This change prints the process name even before trying to open
the file, returning it to previous behaviour.
Thanks to Vincent Bernat for some analysis.
Derek Fawcus [Thu, 15 Jan 2015 13:14:53 +0000 (13:14 +0000)]
'slabtop -o' with stdin not a tty would complain
When the command is executed in one shot mode (-o) with stdin
being something other than a terminal, the tcgetattr() call
would fail, and generate an error message. e.g.:
slabtop: terminal setting retrieval: Inappropriate ioctl for device
Active / Total Objects (% used) : 905319 / 915886 (98.8%)
Craig Small [Sat, 24 Jan 2015 06:11:11 +0000 (17:11 +1100)]
skill: fix command line with signal
If skill was used with a signal number then it would intepret
the command line with last option interpreted twice. This often
confused the program so it just would end up killing nothing.
So this would work:
skill -t pts/0
This would not:
skill -9 -t pts/0
The kill path (in the same file) uses the same logic that has
been introduced here.
Jim Warner [Tue, 28 Oct 2014 05:00:00 +0000 (00:00 -0500)]
top: tweak forest view protections for forking anomaly
A recent commit eliminated the potential for a storage
violation with forest view mode. It occurred when some
program (erroneously?) created a lengthy forking loop.
However, the associated commit message was misleading.
The message implied that an unexpected order following
a sort on start_time was the cause of storage overruns
and a 'char' used to track nesting level only distorts
the display when it goes negative. Actually, the truth
is really just the opposite. Any start_time sort quirk
causes no harm while that 'char' can yield corruption.
Should some child end up sorted ahead of its parent by
way of an extremely unlikely shared start_time the end
result is such a child will be displayed unnested just
like init or kthreadd along with all its own children.
However, if nesting levels exceeded 255 (and became 0)
a massive array overrun could be triggered when such a
task and *all* its children were added to an array for
the second time. Exactly how much storage was violated
depended on the number of children that zeroed process
had spawned (hinted at via either SIGSEGV or SIGABRT).
The earlier commit limited nested levels to 100 so the
root cause of the storage violation was already fixed.
The potential for distorted nesting levels due to sort
on start_time would seem to remain. But it's extremely
unlikely that 2 tasks would share the same start_time.
Even so, a new #define has been introduced which makes
top impervious to the order of tasks such that a qsort
is no longer necessary (providing an init/systemd task
exists & was harvested as the first task by readproc).
It can be utilized if distorted nesting ever becomes a
real issue. But since there is a 5-10% performance hit
with that, we'll continue using start_time as default.
Jim Warner [Thu, 23 Oct 2014 05:00:00 +0000 (00:00 -0500)]
top: provide some protection against forking anomalies
This commit will eliminate a very nasty bug associated
with top's forest view mode. It addresses a potential
SIGSEGV/SIGABRT that was only encountered when another
program (erroneously?) creates a lengthy forking loop.
If the growing list of nested children is sufficiently
fast such that proc_t start_time is duplicated between
children then the sort upon which top relies might not
produce the expected order. That, in turn, could cause
the forest_adds function to initially miss some child.
But that missed child would be caught by forest_create
and eventually would cause our array boundary overrun.
Such overrun occurs when some child of that originally
*missed* child is found and a duplicate add attempted.
In correcting this bug we'll also use this opportunity
to prohibit a borrowed proc_t padding byte (char) from
going negative. If the nesting level exceeded 127, the
effect was an "unnesting" with the snprintf width then
viewed as flag+width also yielding left justification.
Henceforth, we'll limit nesting to 100 with subsequent
children shown as " + ", not the usual " `- " prefix.
When startup defaults were changed users with existing
rcfiles would likely find their previous configuration
was not being honored in all respects. The disparities
involved Graphs modes and Summary/Task memory scaling.
This patch simply restores what was always intended as
the proper behavior for previously saved config files.
References(s):
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762928
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762947
. new startup defaults
commit 8ef6cd91fc5df6372a93bc4a37cbad34ead9654e
Signed-off-by: Jim Warner <james.warner@comcast.net>
Previously the default background color was expected
to be always black and the default foreground color
was expected to be always white. This commit extends
the group of color pairs with pairs containing default
colors.
projects / procps-ng / log