Peter Schiffer [Wed, 11 Feb 2015 18:21:12 +0000 (19:21 +0100)]
Portable way to detect 64 bit OS in configure script
Different architectures have different format of /proc/cpuinfo and also, only
i386 and x86_64 archs support cpu flags. Using "getconf LONG_BIT" command it's
possible to detect 64 bit OS on multiple different architectures. This patch was
successfully tested on i386, x86_64, s390x, ppc64 and aarch64 archs.
Sebastien GODARD [Tue, 27 Jan 2015 11:23:50 +0000 (12:23 +0100)]
Better handling of get_activity_position() return value
Function get_activity_position() can return -1 on error, ie. when
corresponding activity hasn't been found in array.
This return code was sometimes not tested because we knew that the
activity existed. So we directly used the return code as array index.
Strengthen things now: If such an activity is not found then display an
internal error message and exit.
Sebastien GODARD [Fri, 23 Jan 2015 20:33:52 +0000 (21:33 +0100)]
Add new sadc_options variable to configure script
Add a new variable (sadc_options) used by the configure script.
This option enables the user to enter parameters that will be passed to
sadc in the sa1 script. This can typically be used to select optional
activities that should be collected by sadc.
Sebastien GODARD [Wed, 21 Jan 2015 10:02:46 +0000 (11:02 +0100)]
Update sysstat spec file
RPM spec file for sysstat package has been updated:
1) Now use --disable-file-attr option with configure to allow a non
privileged user to build the package,
2) Remove %attr directive which cannot be applied on symlinks listed in
%files section.
Sebastien GODARD [Mon, 19 Jan 2015 20:40:33 +0000 (21:40 +0100)]
Rename --disable-man-group option to --disable-file-attr
This option was originally intended to prevent the configuration process
from setting the group ID for manual pages that were installed.
It is now also used to tell the build process to not set the user and
group of the /var/log/sa directory when it is created.
This option may be useful when creating an RPM package with a non
privileged user.
Sebastien GODARD [Sun, 16 Nov 2014 15:06:31 +0000 (16:06 +0100)]
sadc/sar: New /proc/meminfo fields to memory statistics
Five new fields have been added to memory statistics displayed by sar.
These new fields are part of A_MEMORY activity and will be displayed
using option "-r ALL" with sar.
No layout format incompatible change with this patch: The datafile
created by this version of sar can still be read by a previous version
of sar (though the new fields won't be displayed of course as they are
unknown by this old version).
Various outputs from sadf (JSON, XML, etc.) have been updated to take
into account these new metrics.
Originally suggested by Peter Portante (@portante).
Fix description of %util in iostat and sar manual pages
Description of %util in iostat manual page was not accurate, because it
is not percentage of CPU time, but percentage of elapsed time during
which I/O requesrt were issued to the device.
Also make sar manual page consistent with that of iostat.
sadc: Fix possible race condition in signal handler code
Commit 1b52f939 tried to stop sending SIGINT to init process.
But there may be very small time window between parent ID
validation and sending kill signal to parent. If sar dies
during this window (though the chance is very small) we end up
sending signal to init.
Sebastien GODARD [Sat, 18 Oct 2014 13:35:43 +0000 (15:35 +0200)]
sadc: Don't send signal if parent process is already dead
If sar (parent of sadc) dies before sadc completes its job
then init will become sadc parent. Now if we pass SIGINT to
sadc, it will pass that to init process and result in system
reboot.
This patch check parent process before sending signal.
Test case:
Run sar_test script in shell prompt and press ctrl+C twice.
cat sar_test
while true; do /usr/local/bin/sar -n DEV 1 3; sleep 3; done
Sebastien GODARD [Sat, 18 Oct 2014 13:03:07 +0000 (15:03 +0200)]
Add irqstat command
Add new "irqstat" command (contributed by Lance Shelton from
SanDisk/Fusion-io). This command monitors /proc/interrupts and is
designed for NUMA systems with many processors.
irqstat is currently a Python script that you can find in the contrib
directory.
Sebastien GODARD [Sun, 12 Oct 2014 13:07:26 +0000 (15:07 +0200)]
Add irqtop command
Add new "irqtop" command (contributed par Robert Elliott from HP).
This command monitors differences in /proc/interrupts and
/proc/softirqs per CPU, along with CPU statistics.
irqtop is currently a Perl script that you can find in the contrib
directory.
This patch makes sure that sadc doesn't count items several times.
Items here means CPU, network interfaces, etc.
sadc needs to count these items to allocate its structures for each
selected activity. But there are different activities based on the same
items (eg. A_CPU, A_PWR_CPUFREQ and A_PWR_WGHFREQ are all related to CPU
activities, or A_NET_DEV and A_NET_EDEV are related to network
interfaces). And so, when different activities based on the same
items were selected, the number of these items was counted several
times, which is inefficient.
cifsiostat/nfsiostat: Fix output on single core CPU
Uninitialized variable led to wrong numbers being displayed by
cifsiostat and nfsiostat on single core CPU. On multi core machines
reported numbers are correct.
Fix incomplete sar output when end time crosses 24H boundary
sar output was incomplete when used with a file containing data
from two consecutive days and end time crossed 24 hour boundary.
A typical invocation like:
sar -A -s 21:50:00 -e 01:30:00 -f /path/to/datafile
resulted in CPU statistics being displayed only.
This patch fixes the problem and all selected activities are now
displayed.
Sebastien GODARD [Sat, 16 Aug 2014 14:01:45 +0000 (16:01 +0200)]
Update sadf -H output
Update sadf -H output to:
1) display the date the file was created
2) tell whether this is a genuine sa datafile (ie. not converted from an
old datafile) or not.
Option -c added to sadf: Upgrade an old sa datafile
This patch adds option -c to sadf. This option enables the user to
upgrade ("convert") an old system activity datafile (version 9.1.6 and
later) to the up-to-date format (11.1.1 as of today).
Enter "sadf -c old_datafile >new_datafile".
Scheduling policies are defined in <linux/sched.h>. But old kernels lack
some policies added in more recent ones. So define them in pidstat.h if
they are unknown in <linux/sched.h>.
"sar -1 -f" output was different than "sar -f -1" one.
So now enforce options check more aggressively: You cannot enter a day
offset (eg. -1) together with option -f (which tells here to use the
standard sa datafile).
Sebastien GODARD [Mon, 30 Jun 2014 19:55:50 +0000 (21:55 +0200)]
Remove old ll_s_value() function
This function is no longer used: Workaround is now outdated, and a new
workaround is not needed either since the counters used with this
function cannot go backward. So remove it.
KOSAKI Motohiro [Tue, 24 Jun 2014 02:17:36 +0000 (22:17 -0400)]
workaround for /proc/stat go backward
Since dyn-tick feature was introduced (at kernel 2.6.21),
/proc/stat is unreliable and unstable. Stopped cpu can't provide
reliable stat. kernel try to provide a maximum guess. But it is
not 100% accurate.
Unfortunately, the fields are unsigned. So, stepping backward
can make overflow and some command (e.g. mpstat) show very big
number.
KOSAKI Motohiro [Mon, 23 Jun 2014 23:43:50 +0000 (19:43 -0400)]
remove historical workaround for 32bit /proc/stat
/proc/stat was converted to 64bit at kernel 2.6.5 (at 2004). 10 years later,
a workaround for 32bit /proc/stat become completely outdated. It's time to die.
This patch removes the workaround.
For the record, you can see the kernel commit that changed /proc/stat to 64bit
in the following URL.
http://git.kernel.org/cgit/linux/kernel/git/tglx/history.git/commit/?id=0281d847f3a903ab9d0799c5103416d4893fcb34
Sebastien GODARD [Sun, 22 Jun 2014 14:09:18 +0000 (16:09 +0200)]
Use statvfs() instead of statfs() system call
Use statvfs() system call instead of statfs() to get filesystems
statistics with sar since:
1) statfs() has been deprecated by the LSB (useful only to get fs
type which is not needed here),
2) statvfs() better handles large file sizes.
Sebastien GODARD [Mon, 16 Jun 2014 19:38:07 +0000 (21:38 +0200)]
Starting sysstat 11.1.1
Update sysstat version number in configure script.
Note: 11.1.1 will be a development version, including all the new
features added to sysstat.
Each development version will be associated with a stable version,
including only fixes. So don't worry, there will also actually be a
version 11.0.1, and a version 11.0.2 etc. available for download from my
web site.
Sebastien GODARD [Wed, 11 Jun 2014 19:17:45 +0000 (21:17 +0200)]
pidstat: Add option -R to display priority and policy information
Processes' scheduling priority and policy information are no longer
displayed with context switches stats. Create a dedicated report
instead, which can be displayed using pidstat's new option -R.
You have to be in the same directory to check if program exists and is
executable. Moreover the replacement value was wrong (I must have been
tired when I committed this :-( )
Update the FAQ and various manual pages to document recent changes (new
variables added to sysstat parameters file, new naming convention for
standard system activity daily data and report files, etc.)
No longer use a month-by-month directory structure when HISTORY is
greater than 28
This patch updates sa1 and sa2 shell scripts in order to no longer use
the tree of directories that used to be created in the /var/log/sa
directory when HISTORY was greater than 28.
Now, when HISTORY is greater than 28, sadc is called with option -D to
create data files named saYYYYMMDD. All these files are located in the
same directory.
This patch also updates the sysstat parameters file to add a new
variable: SA_DIR, which gives the directory where sa and sar files are
saved.
Accept new naming convention for standard daily data files
sadc/sar/sadf: The standard daily data files may now be named saYYYYMMDD
instead of saDD, where YYYY stands for the current year, MM for the
current month, and DD for the current day.
Option -D has been added to sadc to tell it to use saYYYYMMDD instead of
saDD. This switch works only if no filename is explicitly given on the
command line with sadc.
The same option -D has been added to sar. It also tells it to use
saYYYYMMDD instead of saDD. This switch works only if no filename is
explicitly given on the command line *and* in conjunction with option
-o (ie. when sar is used to write data to files).
If you use sar to read data from a file without specifying this file's
name on the command line (eg. you enter just "sar") then sar will look
for the most recent of saDD and saYYYYMMDD and use it. sadf will do the
same.
If you use sar to write data to files (using its option -o) then it will
always use saDD data file unless option -D is used (in which case
saYYYYMMDD will be used). This makes it consistent with sadc behavior
(and I think it is safer not to let sar decide which file to write to).
Take into account alternate locations for sa data files
sadc, sar and sadf have been updated to take into account alternate
directory locations for sa daily data files (normally saved in
/var/log/sa directory).
When a directory (instead of a plain file) is used with sadc, sadf and
sar (with its options -f and -o) then it is now considered as the
location where the standard saDD data files are saved.
Sebastien GODARD [Fri, 23 May 2014 12:37:29 +0000 (14:37 +0200)]
pidstat: Add an option to filter processes by name
Add "-G <process_name>" option to display only processes whose
name matches <process_name> regular expression.
While -C option filters both process and thread names, -G option
filters only process names, and displays all threads of matching
processes.
Signed-off-by: Cedric Marie <cedric.marie@openmailbox.org> Signed-off-by: Sebastien GODARD <sysstat@users.noreply.github.com>
Sebastien GODARD [Wed, 21 May 2014 19:17:20 +0000 (21:17 +0200)]
Lower HISTORY limit to 25 for scripts to create multiple directories
This patch lowers HISTORY value limit from 28 to 25. When HISTORY value
is greater than 25 then sa1 script creates a month-by-month directory
structure in /var/log/sa to save datafiles.
This value guarantees that we will always have a full history and that
no files will be overwritten unintentionally.
Sebastien GODARD [Fri, 16 May 2014 20:16:37 +0000 (22:16 +0200)]
Don't install crontabs when using systemd timer units
systemd now has timer units that can replace cron jobs.
Such units are installed by sysstat if requested to do so on machines
supporting systemd. In this case a crontab should no longer be
installed else a redundant line of statistics appear at every interval
of time.
Sebastien GODARD [Wed, 14 May 2014 19:50:54 +0000 (21:50 +0200)]
Prevent sar from appending data to data from the preceding month
Previously, the sar utility set the maximum number of days to be logged
in one month too high. Consequently, data from a month were appended to
data from the preceding month. With this update, the maximum number of
days has been set to 25, and data from a month now correctly replaces
data from the preceding month.
From Red Hat Bugzilla #578929 (Jeff Bastian):
*** BEGIN ***
Although RHEL 4 was modified to set the max history for sar to 26 days,
this value is still too long.
Given that
1. February only has 28 days, and
2. that 'find ... -mtime +26' finds files that were modified 27
or more days ago, and
3. find ignores fractional days and thus has rounding errors, and
4. the DST changes take place within sar's $HISTORY window
(as of 2007, the second Sunday in March[1], or March 14, 2010
this year)
then /usr/lib/sa/sa2 is only deleting files that are 28 days
or older (not 27).
As a result, sar data from March 15 to March 28 was getting appended
to the data from February instead of replacing it. You can see the
files
from March 15 to 28 are 2x the size of normal files.
# ls -al
...
-rw-r--r-- 1 root root 813552 Mar 12 23:50 sa12
-rw-r--r-- 1 root root 813552 Mar 13 23:50 sa13
-rw-r--r-- 1 root root 779664 Mar 14 23:50 sa14
-rw-r--r-- 1 root root 1626864 Mar 15 23:50 sa15
-rw-r--r-- 1 root root 1626864 Mar 16 23:50 sa16
-rw-r--r-- 1 root root 1626864 Mar 17 23:50 sa17
...
-rw-r--r-- 1 root root 1626864 Mar 26 23:50 sa26
-rw-r--r-- 1 root root 1626864 Mar 27 23:50 sa27
-rw-r--r-- 1 root root 1626864 Mar 28 23:50 sa28
-rw-r--r-- 1 root root 813552 Mar 29 23:50 sa29
-rw-r--r-- 1 root root 339120 Mar 30 09:50 sa30
...
The max needs to be 25 days to account for DST now.
Attached is a demo script to show how 'find ... -mtime +26 ...' fails to
find files that are 27 days old when dealing with the DST time change on
March 14, 2010.
It touches a series of files, sa10 to sa20, with timestamps from
February 10 to February 20 at 23:53. It then sets the date to March 15
and finds files that are older than 26 days. (And finally restores the
date.)
Stopping ntpd: [ OK ]
Stopping crond: [ OK ]
Mon Mar 15 23:53:00 EDT 2010
-rw-r--r-- 1 root root 0 Feb 15 23:53 /root/timestamp/sa15
-rw-r--r-- 1 root root 0 Feb 14 23:53 /root/timestamp/sa14
-rw-r--r-- 1 root root 0 Feb 13 23:53 /root/timestamp/sa13
-rw-r--r-- 1 root root 0 Feb 12 23:53 /root/timestamp/sa12
-rw-r--r-- 1 root root 0 Feb 11 23:53 /root/timestamp/sa11
-rw-r--r-- 1 root root 0 Feb 10 23:53 /root/timestamp/sa10
Thu Apr 1 12:35:12 EDT 2010
Starting crond: [ OK ]
Starting ntpd: [ OK ]
You can see that youngest file it found was from February 15 which is 28
days old on March 15. It does not find sa16 which is 27 days (in our
thinking) even though we specified +26. This is due to the rounding
errors from find ignoring fractional parts of days.
*** END ***
Sebastien GODARD [Sat, 15 Mar 2014 07:39:54 +0000 (08:39 +0100)]
Display number of activities saved in file's header
Update sadf so that it can display the total number of activities and
the number of volatile activities in file.
These numbers can be displayed with sadf -H.
Sebastien GODARD [Fri, 14 Mar 2014 10:08:00 +0000 (11:08 +0100)]
sar: Take into account a change of CPU number insar datafile (5)
The goal of the next few patches to come is to make it possible for
other activities than A_CPU (displayed by sar -u) to take into account a
change of CPU count in sar data files. These activities must be directly
related to CPU: At the present time these are A_PWR_CPUFREQ (displayed
by sar -m CPU) and A_PWR_WGHFREQ (displayed by sar -m FREQ).
With previous work done, a change from, say 6 to 8 CPU is taken into
account by sar -u:
$ sar -P ALL -f data
Linux 3.9.10-100 (home) 02/08/2014 _x86_64_ (6 CPU)
1) Changes the format of sar data file (this doesn't hurt since 10.3.1
has already changed it. Done again here just in case someone had cloned
from 10.3.1 though this version has not been officially released).
2) Adds a new field in data file's header (sa_vol_act_nr), giving the
number of activities in file taking into account a change of CPU count.
(Such activities, like A_CPU or A_PWR_CPUFREQ are called "volatile
activities" here).
3) When a restart mark is inserted, sadc also writes a few additional
data giving the new number of CPU for each volatile activity in file.
Of course sar has also been updated to read those data and take into
account a change of CPU count for all volatile activities to display.
Tomasz Torcz [Sat, 1 Mar 2014 12:58:03 +0000 (13:58 +0100)]
add systemd timer units replacing cronjobs
Systemd timer units, while preserving functionality of cronjobs,
provide some additional features:
- dependency on cron can be dropped
- timer units next expiration is clearly visible in "systemctl list-timers"
- unit customization rules provide cleaner way for administrator to modify
default parameters, enable, disable and override individual timers
Mike Kazantsev [Mon, 17 Feb 2014 20:01:26 +0000 (02:01 +0600)]
Fix output of sadf -j with file-utc-time present
Commit aea4561 added "file-utc-time" parameter output (if present in the
datafile), but didn't add comma before it in "sadf -j" output, making it
unparseable. Fix that.
Be more user-friendly when trying to access a non existent data file
When sar and sadf try to access a non existent standard daily data file,
tell the user to check if data collecting is enabled.
By doing so, the user goes to the docs looking for how to enable the
data collection, rather than looking for help on why the software seems
to be broken.
sadf -H now displays the last CPU count recorded in the data file. This
is the value of the last_cpu_nr field saved in file's header, giving the
number of processors the machine had when the last sample of statistics
was appended to the file.
projects / sysstat / log