Sebastien GODARD [Sat, 16 Aug 2014 14:01:45 +0000 (16:01 +0200)]
Update sadf -H output
Update sadf -H output to:
1) display the date the file was created
2) tell whether this is a genuine sa datafile (ie. not converted from an
old datafile) or not.
Option -c added to sadf: Upgrade an old sa datafile
This patch adds option -c to sadf. This option enables the user to
upgrade ("convert") an old system activity datafile (version 9.1.6 and
later) to the up-to-date format (11.1.1 as of today).
Enter "sadf -c old_datafile >new_datafile".
Scheduling policies are defined in <linux/sched.h>. But old kernels lack
some policies added in more recent ones. So define them in pidstat.h if
they are unknown in <linux/sched.h>.
"sar -1 -f" output was different than "sar -f -1" one.
So now enforce options check more aggressively: You cannot enter a day
offset (eg. -1) together with option -f (which tells here to use the
standard sa datafile).
Sebastien GODARD [Mon, 30 Jun 2014 19:55:50 +0000 (21:55 +0200)]
Remove old ll_s_value() function
This function is no longer used: Workaround is now outdated, and a new
workaround is not needed either since the counters used with this
function cannot go backward. So remove it.
KOSAKI Motohiro [Tue, 24 Jun 2014 02:17:36 +0000 (22:17 -0400)]
workaround for /proc/stat go backward
Since dyn-tick feature was introduced (at kernel 2.6.21),
/proc/stat is unreliable and unstable. Stopped cpu can't provide
reliable stat. kernel try to provide a maximum guess. But it is
not 100% accurate.
Unfortunately, the fields are unsigned. So, stepping backward
can make overflow and some command (e.g. mpstat) show very big
number.
KOSAKI Motohiro [Mon, 23 Jun 2014 23:43:50 +0000 (19:43 -0400)]
remove historical workaround for 32bit /proc/stat
/proc/stat was converted to 64bit at kernel 2.6.5 (at 2004). 10 years later,
a workaround for 32bit /proc/stat become completely outdated. It's time to die.
This patch removes the workaround.
For the record, you can see the kernel commit that changed /proc/stat to 64bit
in the following URL.
http://git.kernel.org/cgit/linux/kernel/git/tglx/history.git/commit/?id=0281d847f3a903ab9d0799c5103416d4893fcb34
Sebastien GODARD [Sun, 22 Jun 2014 14:09:18 +0000 (16:09 +0200)]
Use statvfs() instead of statfs() system call
Use statvfs() system call instead of statfs() to get filesystems
statistics with sar since:
1) statfs() has been deprecated by the LSB (useful only to get fs
type which is not needed here),
2) statvfs() better handles large file sizes.
Sebastien GODARD [Mon, 16 Jun 2014 19:38:07 +0000 (21:38 +0200)]
Starting sysstat 11.1.1
Update sysstat version number in configure script.
Note: 11.1.1 will be a development version, including all the new
features added to sysstat.
Each development version will be associated with a stable version,
including only fixes. So don't worry, there will also actually be a
version 11.0.1, and a version 11.0.2 etc. available for download from my
web site.
Sebastien GODARD [Wed, 11 Jun 2014 19:17:45 +0000 (21:17 +0200)]
pidstat: Add option -R to display priority and policy information
Processes' scheduling priority and policy information are no longer
displayed with context switches stats. Create a dedicated report
instead, which can be displayed using pidstat's new option -R.
You have to be in the same directory to check if program exists and is
executable. Moreover the replacement value was wrong (I must have been
tired when I committed this :-( )
Update the FAQ and various manual pages to document recent changes (new
variables added to sysstat parameters file, new naming convention for
standard system activity daily data and report files, etc.)
No longer use a month-by-month directory structure when HISTORY is
greater than 28
This patch updates sa1 and sa2 shell scripts in order to no longer use
the tree of directories that used to be created in the /var/log/sa
directory when HISTORY was greater than 28.
Now, when HISTORY is greater than 28, sadc is called with option -D to
create data files named saYYYYMMDD. All these files are located in the
same directory.
This patch also updates the sysstat parameters file to add a new
variable: SA_DIR, which gives the directory where sa and sar files are
saved.
Accept new naming convention for standard daily data files
sadc/sar/sadf: The standard daily data files may now be named saYYYYMMDD
instead of saDD, where YYYY stands for the current year, MM for the
current month, and DD for the current day.
Option -D has been added to sadc to tell it to use saYYYYMMDD instead of
saDD. This switch works only if no filename is explicitly given on the
command line with sadc.
The same option -D has been added to sar. It also tells it to use
saYYYYMMDD instead of saDD. This switch works only if no filename is
explicitly given on the command line *and* in conjunction with option
-o (ie. when sar is used to write data to files).
If you use sar to read data from a file without specifying this file's
name on the command line (eg. you enter just "sar") then sar will look
for the most recent of saDD and saYYYYMMDD and use it. sadf will do the
same.
If you use sar to write data to files (using its option -o) then it will
always use saDD data file unless option -D is used (in which case
saYYYYMMDD will be used). This makes it consistent with sadc behavior
(and I think it is safer not to let sar decide which file to write to).
Take into account alternate locations for sa data files
sadc, sar and sadf have been updated to take into account alternate
directory locations for sa daily data files (normally saved in
/var/log/sa directory).
When a directory (instead of a plain file) is used with sadc, sadf and
sar (with its options -f and -o) then it is now considered as the
location where the standard saDD data files are saved.
Sebastien GODARD [Fri, 23 May 2014 12:37:29 +0000 (14:37 +0200)]
pidstat: Add an option to filter processes by name
Add "-G <process_name>" option to display only processes whose
name matches <process_name> regular expression.
While -C option filters both process and thread names, -G option
filters only process names, and displays all threads of matching
processes.
Signed-off-by: Cedric Marie <cedric.marie@openmailbox.org> Signed-off-by: Sebastien GODARD <sysstat@users.noreply.github.com>
Sebastien GODARD [Wed, 21 May 2014 19:17:20 +0000 (21:17 +0200)]
Lower HISTORY limit to 25 for scripts to create multiple directories
This patch lowers HISTORY value limit from 28 to 25. When HISTORY value
is greater than 25 then sa1 script creates a month-by-month directory
structure in /var/log/sa to save datafiles.
This value guarantees that we will always have a full history and that
no files will be overwritten unintentionally.
Sebastien GODARD [Fri, 16 May 2014 20:16:37 +0000 (22:16 +0200)]
Don't install crontabs when using systemd timer units
systemd now has timer units that can replace cron jobs.
Such units are installed by sysstat if requested to do so on machines
supporting systemd. In this case a crontab should no longer be
installed else a redundant line of statistics appear at every interval
of time.
Sebastien GODARD [Wed, 14 May 2014 19:50:54 +0000 (21:50 +0200)]
Prevent sar from appending data to data from the preceding month
Previously, the sar utility set the maximum number of days to be logged
in one month too high. Consequently, data from a month were appended to
data from the preceding month. With this update, the maximum number of
days has been set to 25, and data from a month now correctly replaces
data from the preceding month.
From Red Hat Bugzilla #578929 (Jeff Bastian):
*** BEGIN ***
Although RHEL 4 was modified to set the max history for sar to 26 days,
this value is still too long.
Given that
1. February only has 28 days, and
2. that 'find ... -mtime +26' finds files that were modified 27
or more days ago, and
3. find ignores fractional days and thus has rounding errors, and
4. the DST changes take place within sar's $HISTORY window
(as of 2007, the second Sunday in March[1], or March 14, 2010
this year)
then /usr/lib/sa/sa2 is only deleting files that are 28 days
or older (not 27).
As a result, sar data from March 15 to March 28 was getting appended
to the data from February instead of replacing it. You can see the
files
from March 15 to 28 are 2x the size of normal files.
# ls -al
...
-rw-r--r-- 1 root root 813552 Mar 12 23:50 sa12
-rw-r--r-- 1 root root 813552 Mar 13 23:50 sa13
-rw-r--r-- 1 root root 779664 Mar 14 23:50 sa14
-rw-r--r-- 1 root root 1626864 Mar 15 23:50 sa15
-rw-r--r-- 1 root root 1626864 Mar 16 23:50 sa16
-rw-r--r-- 1 root root 1626864 Mar 17 23:50 sa17
...
-rw-r--r-- 1 root root 1626864 Mar 26 23:50 sa26
-rw-r--r-- 1 root root 1626864 Mar 27 23:50 sa27
-rw-r--r-- 1 root root 1626864 Mar 28 23:50 sa28
-rw-r--r-- 1 root root 813552 Mar 29 23:50 sa29
-rw-r--r-- 1 root root 339120 Mar 30 09:50 sa30
...
The max needs to be 25 days to account for DST now.
Attached is a demo script to show how 'find ... -mtime +26 ...' fails to
find files that are 27 days old when dealing with the DST time change on
March 14, 2010.
It touches a series of files, sa10 to sa20, with timestamps from
February 10 to February 20 at 23:53. It then sets the date to March 15
and finds files that are older than 26 days. (And finally restores the
date.)
Stopping ntpd: [ OK ]
Stopping crond: [ OK ]
Mon Mar 15 23:53:00 EDT 2010
-rw-r--r-- 1 root root 0 Feb 15 23:53 /root/timestamp/sa15
-rw-r--r-- 1 root root 0 Feb 14 23:53 /root/timestamp/sa14
-rw-r--r-- 1 root root 0 Feb 13 23:53 /root/timestamp/sa13
-rw-r--r-- 1 root root 0 Feb 12 23:53 /root/timestamp/sa12
-rw-r--r-- 1 root root 0 Feb 11 23:53 /root/timestamp/sa11
-rw-r--r-- 1 root root 0 Feb 10 23:53 /root/timestamp/sa10
Thu Apr 1 12:35:12 EDT 2010
Starting crond: [ OK ]
Starting ntpd: [ OK ]
You can see that youngest file it found was from February 15 which is 28
days old on March 15. It does not find sa16 which is 27 days (in our
thinking) even though we specified +26. This is due to the rounding
errors from find ignoring fractional parts of days.
*** END ***
Sebastien GODARD [Sat, 15 Mar 2014 07:39:54 +0000 (08:39 +0100)]
Display number of activities saved in file's header
Update sadf so that it can display the total number of activities and
the number of volatile activities in file.
These numbers can be displayed with sadf -H.
Sebastien GODARD [Fri, 14 Mar 2014 10:08:00 +0000 (11:08 +0100)]
sar: Take into account a change of CPU number insar datafile (5)
The goal of the next few patches to come is to make it possible for
other activities than A_CPU (displayed by sar -u) to take into account a
change of CPU count in sar data files. These activities must be directly
related to CPU: At the present time these are A_PWR_CPUFREQ (displayed
by sar -m CPU) and A_PWR_WGHFREQ (displayed by sar -m FREQ).
With previous work done, a change from, say 6 to 8 CPU is taken into
account by sar -u:
$ sar -P ALL -f data
Linux 3.9.10-100 (home) 02/08/2014 _x86_64_ (6 CPU)
1) Changes the format of sar data file (this doesn't hurt since 10.3.1
has already changed it. Done again here just in case someone had cloned
from 10.3.1 though this version has not been officially released).
2) Adds a new field in data file's header (sa_vol_act_nr), giving the
number of activities in file taking into account a change of CPU count.
(Such activities, like A_CPU or A_PWR_CPUFREQ are called "volatile
activities" here).
3) When a restart mark is inserted, sadc also writes a few additional
data giving the new number of CPU for each volatile activity in file.
Of course sar has also been updated to read those data and take into
account a change of CPU count for all volatile activities to display.
Tomasz Torcz [Sat, 1 Mar 2014 12:58:03 +0000 (13:58 +0100)]
add systemd timer units replacing cronjobs
Systemd timer units, while preserving functionality of cronjobs,
provide some additional features:
- dependency on cron can be dropped
- timer units next expiration is clearly visible in "systemctl list-timers"
- unit customization rules provide cleaner way for administrator to modify
default parameters, enable, disable and override individual timers
Mike Kazantsev [Mon, 17 Feb 2014 20:01:26 +0000 (02:01 +0600)]
Fix output of sadf -j with file-utc-time present
Commit aea4561 added "file-utc-time" parameter output (if present in the
datafile), but didn't add comma before it in "sadf -j" output, making it
unparseable. Fix that.
Be more user-friendly when trying to access a non existent data file
When sar and sadf try to access a non existent standard daily data file,
tell the user to check if data collecting is enabled.
By doing so, the user goes to the docs looking for how to enable the
data collection, rather than looking for help on why the software seems
to be broken.
sadf -H now displays the last CPU count recorded in the data file. This
is the value of the last_cpu_nr field saved in file's header, giving the
number of processors the machine had when the last sample of statistics
was appended to the file.
Move header_size field to the end of the structure, so that other
programs can still tell it is a sysstat sar data file, even though
the format has changed.
Also, to avoid causing format changes, add padding to the file_magic
structure so that when they grow in size other tools can still
handle the existing data.
sar: Take into account a change of CPU number in sar datafile (3)
Update sar so that it now reads the number of CPU associated with a
RESTART record. Reallocate CPU structures accordingly, and display the
number of CPU with the restart mark.
sar: Take into account a change of CPU number in sar datafile (2)
sadc now accepts a change of CPU count when inserting a restart mark. In
this case, sadc sets the new value in file's header (last_cpu_nr field)
then rewrites it.
The new number of CPU is also written after the restart mark record in
file.
Sebastien GODARD [Sun, 26 Jan 2014 16:28:15 +0000 (17:28 +0100)]
sar: Take into account a change of CPU number in sar datafile (1)
On virtual machine, after CPU change (e.g. power off machine ->
add 1 cpu -> power on machine), sar command doesn't display
previous data (from today) from the current saXX file.
This is the first patch aimed at making sar able to take into account
a change of CPU count in its datafiles.
This patch adds a new field into file's header data (last_cpu_nr)
which gives the number of CPU the machine had when the last sample of
data was appended to the file.
Sar data file format is no longer compatible with that from previous
sysstat versions. To be able to add other fields to file's header
in the future without making it incompatible again, also add a
field (header_size) to file's magic data giving the size of file's
header.
Sebastien GODARD [Sun, 26 Jan 2014 08:17:03 +0000 (09:17 +0100)]
Rename nfsiostat command to nfsiostat-sysstat
nfsiostat was added to the sysstat package in 2010, but such a command
has already existed in the nfs-utils package since 2008.
So to avoid confusion, rename nfsiostat to nfsiostat-sysstat and
indicate it is now obsolete.
The nfsiostat command from the sysstat package will be removed in a
future version since we don't need two similar commands.
Sebastien GODARD [Wed, 15 Jan 2014 21:31:33 +0000 (22:31 +0100)]
Add option --enable-copy-only to configure script
Add a new option (--enable-configure-only) to configure script. This
option makes sure that files are only copied when installing sysstat and
that nothing else (like activating a service for distro using systemd)
is done. This may be useful when creating sysstat package.
Sebastien GODARD [Mon, 13 Jan 2014 21:07:02 +0000 (22:07 +0100)]
sadf now displays file creation time
sadf -H, sadf -x and sadf -j now display the creation time of the
datafile. This time was available in the header of the datafile.
Time is displayed in UTC.
Sebastien GODARD [Sun, 12 Jan 2014 07:10:51 +0000 (08:10 +0100)]
Take $DESTDIR variable into account when installing sysstat service used by systemd.
Previous code for installing sysstat using systemd needed rw permissions
to the root directory. Yet some users and distributions want to install
files into $(DESTDIR) before actually installing into the root directory.
So take $DESTDIR variable into account.
pidstat: Display stats since boot time for a list of given processes
pidstat displays statistics since system startup when the interval and
count parameters are not set on the command line (eg. entering "pidstat
-d" will display I/O statistics for all processes that have had I/O
activity since boot time). But pidstat couldn't display those stats when
some PID numbers were entered on the command line (eg. "pidstat -d -p
1234" to display I/O stats since system startup for process 1234).
This patch makes it possible now.
Rearrange options displayed by sar in its short help message
(sar -h): The upper case option should be displayed before its lower
case counterpart to be consistent with options order displayed by sar
usage message or displayed in sar manual page.
pidstat -d now displays -1 for I/O statistics values when the
file containing the statistics data for the corresponding process cannot
be read (permission denied or file non existent).
pidstat stack utilization statistics were not always properly refreshed
Stack utilization statistics displayed by "pidstat -s" were sometimes
not displayed for some processes although values had changed.
This was because stack stats were displayed together with memory stats,
but pidstat was looking only for a variation in memory counters values
to decide whether to display the corresponding process or not.
So now process memory and stack statistics independently.
mpstat/pidstat exit immediately when SIGINT caught during 1st interval
The mpstat and pidstat commands display their average statistics when
they are interrupted by the user with control-C (SIGINT signal).
But when the signal was caught during the first interval of time (ie.
before any statistics had been displayed), trying to display some
average values is irrelevant. So make them exit immediately in this
case.
sar (or sadc) allocates empty records in the data files it creates so
that it can save statistics for devices (disks, network interfaces,
etc.) that may be added to the system after the file was created.
The drawback is that data files take more space on disk than actually
strictly necessary.
Using the "prealloc" variable with configure (before compiling sysstat),
the user can tell how much space he wants to allocate.
This variable will determine the size of data files created by sar/sadc.
The default value is 1, meaning that some empty records will be
allocated.
A value of 0 means that data files will be the smallest possible.
projects / sysstat / log