David Champion [Sun, 29 Jan 2017 02:47:57 +0000 (18:47 -0800)]
Adds label completion.
A global label hash is added, to which labels are added as they're parsed
from a mailbox file or edited manually by the user. Reference counts are
kept in the hash table so that unused labels are removed from available
completions. Completion is available in the label editor only, but it
may be feasible to add for search expressions if the preceding text ends
with '~y'.
David Champion [Tue, 24 Jan 2017 03:01:50 +0000 (19:01 -0800)]
Add subjectrx command to replace matching subjects with something else.
This lets you define regular expressions-replacement pairs for subject
display. When a Subject: matches the regular expression, the replacement
value will be displayed instead in the message index. Backreferences
are supported.
This is especially nice for simplifying subjects that are overly wordy,
such as mailing list posts (with [Listname] tags, etc), mail from
ticketing systems or bug trackers, etc. It lets you reduce clutter in
your mutt display without altering the messages themselves.
David Champion [Tue, 24 Jan 2017 03:01:36 +0000 (19:01 -0800)]
Abstract the SPAM_LIST as a generic REPLACE_LIST
REPLACE_LIST can be used more generally as a list of pattern
match-replace settings. SPAM_LIST was a special case of this, so
spam handling has been been changed to use REPLACE_LIST instead, and
SPAM_LIST was removed.
A generic function for performing a REPLACE_LIST replacement has
been added in mutt_apply_replace().
Commited by Kevin McCarthy with some buffer overflow fixes in
mutt_apply_replace().
Kevin McCarthy [Tue, 24 Jan 2017 02:46:16 +0000 (18:46 -0800)]
Improve Reply-to vs From comparison when replying. (closes #3909)
Prior to this patch, if the Reply-to mailbox matched the From mailbox,
mutt would always use the From address. This was probably done to
preserve the display name, as the Reply-to address is often missing one.
Unfortunately, there are circumstances where the Reply-to display-name
has significance, such as in ticket 3909.
Change mutt so that it only uses the From address if the Reply-To has
no display-name.
Kevin McCarthy [Wed, 18 Jan 2017 00:09:16 +0000 (16:09 -0800)]
Fix several alias hashtable issues.
Convert to use the strdup keys hash. Addresses can be converted back
and forth from intl to local forms. This frees and recreates a new
addr->mailbox string, resulting in the hash table key being a dangling
pointer.
Change alias hash table insert/remove to ensure the address is in intl
form. The alias menu (previously) converted address entries to local
form when performing a completion. Even with the pointer issue fixed,
the entries may not be removed from the hash if the intl and local
forms are different.
Lastly, there is no reason for the alias menu to manually convert to
local form before writing the address to the output buffer.
rfc822_write_address() has a display parameter that will call
mutt_addr_for_display() instead when set. Change to set the display
parameter and remove the conversion calls.
This last change obviates the first two changes, but they are a good idea
in any case.
Kevin McCarthy [Wed, 18 Jan 2017 00:09:11 +0000 (16:09 -0800)]
Add casecmp and strdup_key flags to hash_create()
Aliases and (in the future), X-Label hashes will require a hash that
strdups the key. Convert the casecmp parameter of hash_create() to a
flags parameter, and add a flag to strdup the keys.
David Champion [Sun, 15 Jan 2017 03:18:45 +0000 (19:18 -0800)]
Allow initial blank lines in local mailboxes.
Some mailbox-creation tools erroneously append a blank line to a file
before appending a UNIXv7-format mail message, resulting in mailboxes
that are intended to be valid "mbox" folders but are not. Notably old
versions of Mailman do this, making archive files that cannot be read by
mutt.
This patch causes mutt to skip leading NLs and CRs when detecting magic.
Kevin McCarthy [Fri, 6 Jan 2017 22:17:10 +0000 (14:17 -0800)]
Convert HASH to be indexable by unsigned int. (see #3905)
Convert the HASH to be usable for either string or unsigned int keys,
so that a uid hash can be added for imap.
To keep hash-usage code disruption to a minimum, this introduces new
create/insert/find/delete functions for the int hash, but keeps the
old function names for string keys.
This implementation makes the key a union. It may have been a better
idea to introduce a whole new structure, but this way allows minimum
changes to and maximum reuse of the existing hash code.
Kevin McCarthy [Thu, 5 Jan 2017 03:45:59 +0000 (19:45 -0800)]
Fix imap server-side search to call uid2msgno() only once. (see #3905)
After performing a UID SEARCH, the results are parsed in
cmd_parse_search(). This was accidentally calling uid2msgno() twice.
Since that function does a linear search, this has a noticable impact
on large search results.
Kevin McCarthy [Tue, 3 Jan 2017 02:08:17 +0000 (18:08 -0800)]
Add a pattern_cache_t to speed up a few repeated matches.
Vincent Lefèvre reported experiencing an index display performance
issue. This occurred with messages containing many recipients. He
had many index color lines containing ~l. The ~l ended up being run
over and over on these messages, resulting in a noticable slowdown
displaying the index.
This patch adds caching for just a few of the pattern operations (~l,
~u, ~p, ~P) that are potentially expensive and also don't have
arguments. The caching is only enabled for operations repeatedly
matching against the same message: color, hooks, scoring.
The caching is fairly targeted, but isn't that invasive or
complicated.
Kevin McCarthy [Sun, 1 Jan 2017 03:57:07 +0000 (19:57 -0800)]
Canonicalize line endings for GPGME S/MIME encryption. (closes #3904)
This matches the behavior for S/MIME classic mode: OpenSSL converts
the line endings to cr/lf before encrypting. Although Mutt always
canonicalizes the line endings before verifying the signature, some
clients do not do this for encrypted messages.
Michał Kępień [Tue, 13 Dec 2016 19:16:10 +0000 (11:16 -0800)]
Rework OpenSSL certificate verification to support alternative chains. (closes #3903)
The way Mutt currently verifies SSL certificates using OpenSSL does
not support alternative chains, which may cause confusion when some
popular mail providers (e.g. Gmail) are used with specific sets of
trusted CA certificates.
Replace the "manual" verification done by mutt in
check_certificate_by_signer() with SSL_set_verify() using a callback.
OpenSSL then does the certificate verification, including properly
looking at alternative chains. The callback still provides the
opportunity to override using ~/.mutt_certificates or an interactive
prompt.
Kevin McCarthy [Sat, 3 Dec 2016 23:24:32 +0000 (15:24 -0800)]
Change sidebar_spoolfile coloring to be lower precedence.
Give sidebar_new and sidebar_flagged higher precedence than
sidebar_spoolfile, so that new and flagged message colors will show up
for the spoolfile in the sidebar.
Don't use X509_NAME_oneline() with a fixed size buffer, which could
truncate the string, perhaps leaving off the CN field entirely.
Instead, work directly off the X509_NAME.
Rather than use strstr to tokenize it, call
X509_NAME_get_text_by_NID() with the nid types. Although
X509_NAME_get_text_by_NID() is "legacy", it is the most directly
useful for mutt in this simple interactive prompt.
The function was set up to include the ST and C fields in the prompt,
but the loop limit was too low. I believe this was an oversight, so
increase the loop to include those two fields.
Kevin McCarthy [Mon, 21 Nov 2016 00:19:17 +0000 (16:19 -0800)]
Don't close stderr when opening a tunnel. (closes #3726)
Instead of closing stderr, redirect it to /dev/null in
tunnel_socket_open(). Otherwise a program can accidentally open a
file into handle 2 and then unknowingly use that when trying to print
to stderr.
Thanks to lotheac for the original patch, which I just modified
slightly.
Kevin McCarthy [Sun, 20 Nov 2016 03:35:07 +0000 (19:35 -0800)]
More openssl1.1 fixes: remove uses of X509->name in debugging. (closes #3870)
X509->name was a shortcut for the longer
name = X509_NAME_oneline (X509_get_subject_name (cert),
buf, sizeof (buf));
invocation. Change the debugging to print the cert name and chain
names in the ssl_check_certificate() loop instead.
Kevin McCarthy [Sun, 20 Nov 2016 03:35:07 +0000 (19:35 -0800)]
More openssl1.1 fixes: remove uses of X509->name in debugging. (closes #3870)
X509->name was a shortcut for the longer
name = X509_NAME_oneline (X509_get_subject_name (cert),
buf, sizeof (buf));
invocation. Change the debugging to print the cert name and chain
names in the ssl_check_certificate() loop instead.
Kevin McCarthy [Fri, 18 Nov 2016 23:54:27 +0000 (15:54 -0800)]
Fix mark-message translation and keybind menu.
Move the OP_MARK_MESSAGE and hotkey macro from MENU_GENERIC to the
MENU_MAIN keymap. Putting the macro under generic prevents it from
overriding a keybinding in the index (even if the function is bound to
noop). Additionally, the macro can only be executed from the index,
so it doesn't make sense as a generic keybinding.
Use the term "hotkey" in both the OPS and km_bind description.
Mark the km_bind description translatable.
Add L10N messages for the new translation strings.
Kevin McCarthy [Fri, 18 Nov 2016 01:54:35 +0000 (17:54 -0800)]
Revert changes made to mutt_find_cfg() in 3c6d322912e3
The usage of MUTT_VERSION in mutt_find_cfg() was fine before the
commit: it wasn't using MUTT_VERSION inlined into a string with format
string substitution.
Revert to the version of mutt_find_cfg() before that changeset.
David Champion [Thu, 17 Nov 2016 23:57:42 +0000 (15:57 -0800)]
Adds <mark-message> binding to create "hotkeys" for messages.
<mark-message>foo<enter> will create a new macro "'foo" which will
return to the current message by searching for that message's
message id. The initial character of the macro is defined by
$mark_macro_prefix, and defaults to "'" for verisimilitude vs. vi.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:36 +0000 (20:02 -0800)]
Compress: check mailbox type if possible when appending.
If an append hook isn't defined, then mutt has to decompress the
mailbox. In that case, we can check the type of the decompressed
mailbox instead of defaulting to DefaultMagic.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:35 +0000 (20:02 -0800)]
Compress: fix check_mailbox and sync_mailbox.
Change check_mailbox to delegate to the child_ops->check_mailbox if
the compressed mailbox has changed. This allows the mailbox to
properly recover if both the decompressed mailbox and compressed file
have changed.
Change sync_mailbox to call check_mailbox before attempting to sync.
This will prevent overwriting external changes to the compressed
mailbox.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:35 +0000 (20:02 -0800)]
Create mx_ops.sync operation. Refactor compress to use the mx_ops.sync.
Change compress.sync_mailbox() to lock the compressed mailbox around
both the tempfile sync and compress operations. This will prevent
changes made inbetween the two syncs from being overwritten.
Thanks to Damien Riegel for his original patch refactoring
mx_ops.sync, which this patch is partially based upon.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:35 +0000 (20:02 -0800)]
Compress: pull the lock/unlock operations into the open,close,sync operations.
Some operations, such as open_append and sync, need an exclusive lock
across a longer period than a single compress/decompress. Remove it
from the execute_command and pull into the outer callers. Store lock
information inside compress_info.
Sync and check_mailbox need more fixes, which will be addressed in
subsequent patches.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:34 +0000 (20:02 -0800)]
Compress: add delegate calls to open_append and close mx_ops functions.
The open_append and close were partially duplicating mbox open_append
and close operations internally. Change it to call the actual
delegate functions instead.
Inline the open_read() function inside open_mailbox(). Having it
split improved nothing and just complicated the code (i.e. added
error-handling checks in open_mailbox() that in reality could not
fail).
Kevin McCarthy [Mon, 14 Nov 2016 04:02:34 +0000 (20:02 -0800)]
Compress: fix several logic and memory bugs.
setup_paths leaks memory: realpath is already set in mx_open_mailbox()
restore_paths is unneeded. mx_fastclose_mailbox() will free stuff,
and nothing is looking at the path once we are closing or aborting.
Make a copy of the hooks. Otherwise 'unhook *' will leave dangling
pointers.
Add compress_info freeing inside mx_fastclose_mailbox(). Only free
inside compress.c when we want to prevent close() from doing anything.
close_mailbox() didn't preserve ctx->path on error.
execute_command() didn't return an error if the mutt_system() command
failed.
mx_open_mailbox_append() should check mutt_comp_can_append() only for
the case that the mailbox doesn't exist. When it exists,
mx_get_magic() has already looked at the file contents before checking
for matching open_hooks.
In open_append_mailbox() if no append hook is defined, it should't
call ci->open() if the mailbox doesn't exist. It should act just like
append and create a temporary file.
check_mailbox() needs more work. For now, at least have it properly
close the mailbox on error.
Kevin McCarthy [Mon, 14 Nov 2016 04:02:34 +0000 (20:02 -0800)]
Compress: remove buffy stats "improvisation".
Mutt doesn't support polling compressed mailboxes. This code creates
a false impression and dirties mutt_save_message() in the process. I
don't like it, so am taking it out.
Damien Riegel [Mon, 14 Nov 2016 02:45:35 +0000 (18:45 -0800)]
search muttrc file according to XDG Base Specification (closes #3207)
First of all, the MUTT_VERSION symbol is now concatenated (when
possible) at compile time.
Then, the logic to find the config file has been changed a bit to remove
unnecessary calls to access(), so now each possible locations for the
config file is only tested once, and it stops as soon as a valid one has
been found. So instead of:
access("/home/dkc/.muttrc-1.7.1", F_OK) = -1 ENOENT (No such file or directory)
access("/home/dkc/.muttrc", F_OK) = 0
access("/home/dkc/.muttrc", F_OK) = 0
access("/home/dkc/.muttrc", F_OK) = 0
[... Tests for Muttrc ... ]
access("/home/dkc/.muttrc", F_OK) = 0
We now have:
access("/home/dkc/.muttrc-1.7+13 (f658e517960e)", F_OK) = -1 ENOENT (No such file or directory)
access("/home/dkc/.muttrc", F_OK) = 0
It also cleans up the case where -F is passed on the command line but
points to a non-existent file by moving the error path closer to the
actual fail condition.
Finally, it adds partial support for the XDG Base Directory
Specification. mutt will now try to locate its config at:
The mx_fastclose_mailbox() calls mx_ops->close(), which invokes
mbox_close_mailbox().
Also, close the ctx->fp inside mbox_close_mailbox(). This way, the
(to be added) compress logic can call the mx_ops->close() instead of
"knowing" to close the fp before recompressing.
mx_fastclose_mailbox() will safe_fclose() the fp again, but I'm
leaving it there just in case I missed a usage of the fp in some other
part of the code.
Kevin McCarthy [Tue, 8 Nov 2016 02:10:27 +0000 (18:10 -0800)]
Chain %d->%F->%f in the attachment menu.
Previously, %d would use %f if there was no description set.
Place the new %F option in between %d and %f. This way, %d will fall
back on %F, which will fall back on %f. This allows the standard
attachment menu to show d_filename.
This is useful for forwarding attachments or editing draft files with
attachments. In these cases the actual filename is sanitized but the
attachment name is preserved in d_filename.
Damien Riegel [Tue, 8 Nov 2016 02:10:22 +0000 (18:10 -0800)]
compose: add operation to rename an attachment
As opposed to rename-file, which actually renames the underlying file
of the attachment, rename-attachment puts a value in d_filename, which
is used in the Content-Disposition header.
Kevin McCarthy [Sun, 23 Oct 2016 22:11:56 +0000 (15:11 -0700)]
Perform charset conversion on text attachments when piping. (closes #3773) (see #3886)
When piping a text attachment, there is no reliable way to know the
charset used.
Vincent Lefèvre says:
It was decided in the past that when there is no information on the
charset in a transmission to an external command (e.g. as for mail
composing), texts are expected to be transmitted in the local
charset.
Add a MUTT_CHARSET flag to enable charset conversion on text
attachments for both when $attach_split is set and unset.
Kevin McCarthy [Sun, 23 Oct 2016 21:46:37 +0000 (14:46 -0700)]
Fix pager segfault when lineInfo.chunks overflows. (closes #3888)
The reporter had an html attachment with extremely long lines,
combined with a color pattern of "color body default default ."
This overflowed the lineInfo.chunks, causing a segfault.
Kevin McCarthy [Wed, 19 Oct 2016 20:21:16 +0000 (13:21 -0700)]
Add root-message function to jump to root message in thread.
This seems like a useful feature that was brought up for discussion on
mutt-users. Proposed solutions involved collapsing/uncollapsing
threads, but it's not hard to modify the mutt_parent_message()
function to return the root instead.
Kevin McCarthy [Mon, 17 Oct 2016 18:22:13 +0000 (11:22 -0700)]
Actually fix gpgme segfault in create_recipient_set().
Changeset 6e44bfa16096 did not fix the segv. (Sorry, I made the fix
based off a report on IRC but didn't trigger the segv myself: it was
caused by an out-of-tree patch).
The actual problem was that the rset was only resized on a successful
gpgme_get_key(). However, on error, the array still needs to be
NULL-terminated before calling free_recipient_set().
Move the resize so it always takes place. This obviates the need for
the NULL check added in 6e44bfa16096.
Kevin McCarthy [Sun, 16 Oct 2016 22:44:17 +0000 (15:44 -0700)]
Change sidebar to only match $folder prefix on a $sidebar_divider_char. (closes #3887)
The reporter had a $spoolfile of ~/Mailbox and a $folder of ~/Mail.
The sidebar was truncating the spoolfile to "ox" because it only
looked at a substring prefix match.
Kevin McCarthy [Sun, 16 Oct 2016 21:16:47 +0000 (14:16 -0700)]
Fix gpgme segfault in create_recipient_set().
If gpgme_get_key() errors on the first key, the rset will not be
allocated yet. Attempting to null-terminate (and then free) the array
causes a segfault.