Explain the difference between NL and NEWLINE.

diff --git a/Doc/lib/libtokenize.tex b/Doc/lib/libtokenize.tex
index 3c5e8ef0fe58c0d08fc75ce464c5ec69dec8a667..205407c0202f4d02618e7e0f7584f617f2b3307a 100644 (file)
--- a/Doc/lib/libtokenize.tex
+++ b/Doc/lib/libtokenize.tex
@@ -43,5 +43,8 @@ passed to the \var{tokeneater} function by \function{tokenize()}:
   Token value used to indicate a comment.
 \end{datadesc}
 \begin{datadesc}{NL}
-  Token value used to indicate a newline.
+  Token value used to indicate a non-terminating newline.  The NEWLINE
+  token indicates the end of a logical line of Python code; NL tokens
+  are generated when a logical line of code is continued over multiple
+  physical lines.
 \end{datadesc}