Fix misparsing of non-newline-terminated pg_hba.conf files.

This back-patches the v10-cycle commit 1e5a5d03d into 9.3 - 9.6.
I had noticed at the time that that was fixing a bug, namely that
next_token() might advance *lineptr past the line-terminating '\0',
but given the lack of field complaints I too easily convinced myself
that the problem was only latent.  It's not, because tokenize_file()
decides whether there's more on the line using "strlen(lineptr)".

The bug is indeed latent on a newline-terminated line, because then
the newline-stripping bit in tokenize_file() means we'll have two
or more consecutive '\0's in the buffer, masking the fact that we
accidentally advanced over the first one.  But the last line in
the file might not be null-terminated, allowing the loop to see
and process garbage, as reported by Mark Jones in bug #14859.

The bug doesn't exist in <= 9.2; there next_token() is reading directly
from a file, and termination of the outer loop relies on an feof() test
not a buffer pointer check.  Probably commit 7f49a67f9 can be blamed
for this bug, but I didn't track it down exactly.

Commit 1e5a5d03d does a bit more than the minimum needed to fix the
bug, but I felt the rest of it was good cleanup, so applying it all.

Discussion: https://postgr.es/m/20171017141814.8203.27280@wrigleys.postgresql.org

diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index aa8135eb112e6382a388ab984c77fd91e3f56ff6..c634fa2b7fb4aeef12d872dc784dda192e89941c 100644 (file)
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -141,42 +141,32 @@ next_token(char **lineptr, char *buf, int bufsz, bool *initial_quote,
 {
        int                     c;
        char       *start_buf = buf;
-       char       *end_buf = buf + (bufsz - 2);
+       char       *end_buf = buf + (bufsz - 1);
        bool            in_quote = false;
        bool            was_quote = false;
        bool            saw_quote = false;
 
-       /* end_buf reserves two bytes to ensure we can append \n and \0 */
        Assert(end_buf > start_buf);
 
        *initial_quote = false;
        *terminating_comma = false;
 
-       /* Move over initial whitespace and commas */
+       /* Move over any whitespace and commas preceding the next token */
        while ((c = (*(*lineptr)++)) != '\0' && (pg_isblank(c) || c == ','))
                ;
 
-       if (c == '\0' || c == '\n')
-       {
-               *buf = '\0';
-               return false;
-       }
-
        /*
-        * Build a token in buf of next characters up to EOF, EOL, unquoted comma,
-        * or unquoted whitespace.
+        * Build a token in buf of next characters up to EOL, unquoted comma, or
+        * unquoted whitespace.
         */
-       while (c != '\0' && c != '\n' &&
+       while (c != '\0' &&
                   (!pg_isblank(c) || in_quote))
        {
                /* skip comments to EOL */
                if (c == '#' && !in_quote)
                {
-                       while ((c = (*(*lineptr)++)) != '\0' && c != '\n')
+                       while ((c = (*(*lineptr)++)) != '\0')
                                ;
-                       /* If only comment, consume EOL too; return EOL */
-                       if (c != '\0' && buf == start_buf)
-                               (*lineptr)++;
                        break;
                }
 
@@ -188,12 +178,12 @@ next_token(char **lineptr, char *buf, int bufsz, bool *initial_quote,
                           errmsg("authentication file token too long, skipping: \"%s\"",
                                          start_buf)));
                        /* Discard remainder of line */
-                       while ((c = (*(*lineptr)++)) != '\0' && c != '\n')
+                       while ((c = (*(*lineptr)++)) != '\0')
                                ;
                        break;
                }
 
-               /* we do not pass back the comma in the token */
+               /* we do not pass back a terminating comma in the token */
                if (c == ',' && !in_quote)
                {
                        *terminating_comma = true;
@@ -221,8 +211,8 @@ next_token(char **lineptr, char *buf, int bufsz, bool *initial_quote,
        }
 
        /*
-        * Put back the char right after the token (critical in case it is EOL,
-        * since we need to detect end-of-line at next call).
+        * Un-eat the char right after the token (critical in case it is '\0',
+        * else next call will read past end of string).
         */
        (*lineptr)--;