Add new configuration option --month-report to be able to build monthly
incremental reports.
By default pgBadger in incremental mode only compute daily and weekly reports.
If you want monthly cumulative reports you will have to use a separate command
to specify the report to build. For example to build a report for August 2019:
this will add a link to the month name into the calendar view of incremental
reports to look at monthly report. The report for a current month can be run
every day it is entirely rebuilt each time. The monthly report is not built by
default because it could take lot of time following the amount of data.
Like with --rebuild option, if you use per database report you must add
the -E option to the command.
Thanks to hvisage and Deyu Tian for the feature request.
Add support to RDS and CloudWatch log format, use -f rds if pgbadger is not able to auto-detect this log format. Thanks to peruuparkar and bhushanuparkar2 for the feature request.
Fix option -f | --format that was not applied on all files get from the parameter list where log format auto-detection was failing, the format was taken from the fist file parsed. Thanks to Levente Birta for the report.
Gilles Darold [Wed, 28 Aug 2019 07:19:27 +0000 (09:19 +0200)]
Limit height display size of top queries to avoid taking the whole page
with huge queries. Thanks to ilias ilisepe1 for the patch.
Fix overflow of queries and detail in Slowest individual queries.
Gilles Darold [Fri, 23 Aug 2019 13:48:44 +0000 (15:48 +0200)]
Add report of top N queries that consume most time in the prepare or
parse stage.
Add report of top N queries that consume most time in the bind stage.
Gilles Darold [Thu, 22 Aug 2019 20:11:54 +0000 (22:11 +0200)]
Add report of timing for prepare/bind/execute queries parts. Reported
in a new "Duration" tab in Global Stats report. Example:
Total query duration: 6m16s
Prepare/parse total duration: 45s564ms
Bind total duration: 4m46s
Execute total duration: 44s71m
This also fix previous report of "Total query duration" that was only
reporting execute total duration.
Gilles Darold [Thu, 1 Aug 2019 21:20:42 +0000 (23:20 +0200)]
Revert commit 5e88446, with very logn queries this was worth than
truncating at --maxlength (10Kb by default). Thanks to Bhushan Uparkar
for the report.
835231a / PR #500 did break again the use of wildcards in SSH URI filenames that 0a45659 / #477 took care of. This patch should finally offer a solution that can deal with absolute and relative paths, spaces in paths, wildcards in filenames, single files and whole directories.
Gilles Darold [Tue, 18 Jun 2019 08:22:58 +0000 (10:22 +0200)]
When a query is > 10Kb we first limit size of all constant string parameters
to 30 charaters and then the query is truncated to 10Kb. This prevent
pgbadger to waste time/hang with very long queries when inserting bytea
for example. The 10Kb limit can be controled witgh the --maxlength
command line parameter.
Gilles Darold [Sun, 16 Jun 2019 21:14:01 +0000 (23:14 +0200)]
Allow collect of bind query generating temporary files. Until now only
temporary file statistics was reported but not the query generating
these file. This collect is possible only if log_connection and
log_disconnection are activated in postgresql.conf. Thanks to Ildefonso
Camargo for the feature request.
Luca Ferrari [Tue, 4 Jun 2019 06:27:10 +0000 (08:27 +0200)]
Remove wrong comment preventing the program to exit if no logs are there.
See discussion
<https://github.com/darold/pgbadger/pull/501#issuecomment-498476989>
I should have introduced this comment to prevent the program to exit while
testing.
Gilles Darold [Mon, 3 Jun 2019 02:42:56 +0000 (04:42 +0200)]
Add report per database support to incremental mode. In this mode there
will be a sub directory per database with dedicated incremental reports
and time based sub directory storing the binary statistic data.
Add -H, --html-dir option to be able to set a different path where HTML
report must be written in incremental mode. Binary files remain on directory
defined with -O, --outdir option.
Gilles Darold [Sun, 2 Jun 2019 13:45:06 +0000 (15:45 +0200)]
Add -E | --explode option to explode the main report into one report
per database. Global information not related to a database are added
to the postgres database report.
Update regression test to works with new structure introduced with the
per database report feature.
Luca Ferrari [Fri, 24 May 2019 12:48:54 +0000 (14:48 +0200)]
Implement begin/end hours without the date.
See #495
The idea is to check the '--begin' and '--end' parameters to see if they
represent only a time instead of a timestamp, and then accept them placing the
value into '$from_hour' or '$to_hour' internal variables.
Then, in 'skip_unwanted_lines' check if the time of the line is not between
the limits, similarly to what is done with the timestamp check.
Return the same values as the timestamp check.
Luca Ferrari [Fri, 24 May 2019 12:04:41 +0000 (14:04 +0200)]
Fix remote paths when using SSH.
When specifying a directory instead of a remote file, the program aborts
with a message like:
% pgbadger ssh://postgres@pghost//postgres/data/log/
FATAL: can't get size of remote file,
please check what's going wrong with command:
ssh postgres@pghost "ls -l postgresql-Fri.log"
The problem is that the set_file_list function does not include the path
on the remote host, managing each file path as relative. Therefore
now there is an addition of $dirs before pushing the path to the @lfiles array.
However, this is due only when there is a directory, that is when the number
of remote files on each function invocation is not equal at one.