Fred Drake [Tue, 29 May 2001 15:13:00 +0000 (15:13 +0000)]
Do not start API descriptions with "Does the same, but ..." -- actually
state *which* other function the current one is like, even if the
descriptions are adjacent.
Revise the _PyTuple_Resize() description to reflect the removal of the
third parameter.
Tim Peters [Tue, 29 May 2001 04:27:01 +0000 (04:27 +0000)]
Patch from Gordon McMillan.
updatecache(): When using imputil, sys.path may contain things other than
strings. Ignore such things instead of blowing up.
Hard to say whether this is a bugfix or a feature ...
Thomas Wouters [Mon, 28 May 2001 13:11:02 +0000 (13:11 +0000)]
_PyTuple_Resize: take into account the empty tuple. There can be only one.
Instead of raising a SystemError, just create a new tuple of the desired
size.
Tim Peters [Sun, 27 May 2001 07:39:22 +0000 (07:39 +0000)]
Implement an old idea of Christian Tismer's: use polynomial division
instead of multiplication to generate the probe sequence. The idea is
recorded in Python-Dev for Dec 2000, but that version is prone to rare
infinite loops.
The value is in getting *all* the bits of the hash code to participate;
and, e.g., this speeds up querying every key in a dict with keys
[i << 16 for i in range(20000)] by a factor of 500. Should be equally
valuable in any bad case where the high-order hash bits were getting
ignored.
Also wrote up some of the motivations behind Python's ever-more-subtle
hash table strategy.
Jack Jansen [Sat, 26 May 2001 20:01:41 +0000 (20:01 +0000)]
When reading from stdin (with the dialog box) use any partial line on
stdout as the prompt. This makes raw_input() and print "xxx", ; sys.stdin.readline() work a bit more palatable.
Tim Peters [Sat, 26 May 2001 05:28:40 +0000 (05:28 +0000)]
roundupsize() and friends: fiddle over-allocation strategy for list
resizing.
Accurate timings are impossible on my Win98SE box, but this is obviously
faster even on this box for reasonable list.append() cases. I give
credit for this not to the resizing strategy but to getting rid of integer
multiplication and divsion (in favor of shifting) when computing the
rounded-up size.
For unreasonable list.append() cases, Win98SE now displays linear behavior
for one-at-time appends up to a list with about 35 million elements. Then
it dies with a MemoryError, due to fatally fragmented *address space*
(there's plenty of VM available, but by this point Win9X has broken user
space into many distinct heaps none of which has enough contiguous space
left to resize the list, and for whatever reason Win9x isn't coalescing
the dead heaps). Before the patch it got a MemoryError for the same
reason, but once the list reached about 2 million elements.
Haven't yet tried on Win2K but have high hopes extreme list.append()
will be much better behaved now (NT & Win2K didn't fragment address space,
but suffered obvious quadratic-time behavior before as lists got large).
For other systems I'm relying on common sense: replacing integer * and /
by << and >> can't plausibly hurt, the number of function calls hasn't
changed, and the total operation count for reasonably small lists is about
the same (while the operations are cheaper now).
Fred Drake [Fri, 25 May 2001 04:24:37 +0000 (04:24 +0000)]
Add descriptions of {}.iteritems(), {}.iterkeys(), and {}.itervalues()
in the table of mapping object operations. Re-numbered the list of
notes to reflect the move of the "Added in version 2.2." note to the list
of notes instead of being inserted into the last column of the table.
Tim Peters [Thu, 24 May 2001 16:26:40 +0000 (16:26 +0000)]
dictresize(): Rebuild small tables if there are any dummies, not just if
they're entirely full. Not a question of correctness, but of temporarily
misplaced common sense.
Tim Peters [Wed, 23 May 2001 23:33:57 +0000 (23:33 +0000)]
Jack Jansen hit a bug in the new dict code, reported on python-dev.
dictresize() was too aggressive about never ever resizing small dicts.
If a small dict is entirely full, it needs to rebuild it despite that
it won't actually resize it, in order to purge old dummy entries thus
creating at least one virgin slot (lookdict assumes at least one such
exists).
Also took the opportunity to add some high-level comments to dictresize.
Barry Warsaw [Wed, 23 May 2001 16:59:45 +0000 (16:59 +0000)]
write(): Do two levels of sorting: first sort the individual location
tuples by filename/lineno, then sort the catalog entries by their
location tuples.
Guido van Rossum [Wed, 23 May 2001 13:24:30 +0000 (13:24 +0000)]
When Tim untabified this file, his editor accidentally assumed 4-space
tabs. The title was centered using 8-byte tabs, however, and the
result looked strange. Fixed this.
Tim Peters [Wed, 23 May 2001 07:46:36 +0000 (07:46 +0000)]
Remove test_doctest's expected-output file.
Change test_doctest and test_difflib to pass regrtest's notion of
verbosity on to doctest.
Add explanation for a dozen "new" things to test/README.
Tim Peters [Wed, 23 May 2001 01:45:19 +0000 (01:45 +0000)]
Remove test_difflib's output file and change test_difflib to stop
generating it. Since this is purely a doctest, the output file never
served a good purpose.
Jack Jansen [Tue, 22 May 2001 21:56:42 +0000 (21:56 +0000)]
Lots more Carbon/Carbon.h includes, new UPP routine names, function prototypes. Most toolbox modules now compile, link and import in MacOSX-MachO python.
Tim Peters [Tue, 22 May 2001 20:40:22 +0000 (20:40 +0000)]
SF patch #425242: Patch which "inlines" small dictionaries.
The idea is Marc-Andre Lemburg's, the implementation is Tim's.
Add a new ma_smalltable member to dictobjects, an embedded vector of
MINSIZE (8) dictentry structs. Short course is that this lets us avoid
additional malloc(s) for dicts with no more than 5 entries.
The changes are widespread but mostly small.
Long course: WRT speed, all scalar operations (getitem, setitem, delitem)
on non-empty dicts benefit from no longer needing NULL-pointer checks
(ma_table is never NULL anymore). Bulk operations (copy, update, resize,
clearing slots during dealloc) benefit in some cases from now looping
on the ma_fill count rather than on ma_size, but that was an unexpected
benefit: the original reason to loop on ma_fill was to let bulk
operations on empty dicts end quickly (since the NULL-pointer checks
went away, empty dicts aren't special-cased any more).
Special considerations:
For dicts that remain empty, this change is a lose on two counts:
the dict object contains 8 new dictentry slots now that weren't
needed before, and dict object creation also spends time memset'ing
these doomed-to-be-unsused slots to NULLs.
For dicts with one or two entries that never get larger than 2, it's
a mix: a malloc()/free() pair is no longer needed, and the 2-entry case
gets to use 8 slots (instead of 4) thus decreasing the chance of
collision. Against that, dict object creation spends time memset'ing
4 slots that aren't strictly needed in this case.
For dicts with 3 through 5 entries that never get larger than 5, it's a
pure win: the dict is created with all the space they need, and they
never need to resize. Before they suffered two malloc()/free() calls,
plus 1 dict resize, to get enough space. In addition, the 8-slot
table they ended with consumed more memory overall, because of the
hidden overhead due to the additional malloc.
For dicts with 6 or more entries, the ma_smalltable member is wasted
space, but then these are large(r) dicts so 8 slots more or less doesn't
make much difference. They still benefit all the time from removing
ubiquitous dynamic null-pointer checks, and get a small benefit (but
relatively smaller the larger the dict) from not having to do two
mallocs, two frees, and a resize on the way *to* getting their sixth
entry.
All in all it appears a small but definite general win, with larger
benefits in specific cases. It's especially nice that it allowed to
get rid of several branches, gotos and labels, and overall made the
code smaller.
Fred Drake [Tue, 22 May 2001 19:36:50 +0000 (19:36 +0000)]
Per discussion with Barry, make the default value for both get() and
setdefault() the empty string. In setdefault(), use + to join the value
to create the entry for the headers attribute so that TypeError is raised
if the value is of the wrong type.
Tim Peters [Tue, 22 May 2001 18:28:25 +0000 (18:28 +0000)]
Implementing an idea from Guido on the checkins list:
When regrtest.py finds an attribute "test_main" in a test it imports,
regrtest runs the test's test_main after the import. test_threaded_import
needs this else the cross-thread import lock prevents it from making
progress. Other tests can use this hack too, but I doubt it will ever be
popular.
Guido van Rossum [Tue, 22 May 2001 16:48:37 +0000 (16:48 +0000)]
file_getiter(): make iter(file) be equivalent to file.xreadlines().
This should be faster.
This means:
(1) "for line in file:" won't work if the xreadlines module can't be
imported.
(2) The body of "for line in file:" shouldn't use the file directly;
the effects (e.g. of file.readline(), file.seek() or even
file.tell()) would be undefined because of the buffering that goes
on in the xreadlines module.
Fred Drake [Tue, 22 May 2001 15:12:46 +0000 (15:12 +0000)]
Update to add get() and setdefault() as supported mapping operations, and
add a list of the mapping methods which are not supported (per Barry's
comments).
Jack Jansen [Tue, 22 May 2001 14:13:02 +0000 (14:13 +0000)]
Fixed a nasty slowdown in imports in frozen applications: the shortcut
for loading modules from the application resource fork stopped working
when sys.path component normalization was implemented. Comparison
of sys.path components is now done by FSSpec in stead of by pathname.
Tim Peters [Tue, 22 May 2001 09:34:27 +0000 (09:34 +0000)]
New test adapted from the ancient Demo/threads/bug.py.
ICK ALERT: read the long comment block before run_the_test(). It was
almost impossible to get this to run without instant deadlock, and the
solution here sucks on several counts. If you can dream up a better way,
let me know!
Tim Peters [Tue, 22 May 2001 06:54:14 +0000 (06:54 +0000)]
Changed all the examples with ugly platform-dependent float output to use
numbers that display nicely after repr(). From much doctest experience
with the same trick, I believe people find examples with simple fractions
easier to understand too: they can usually check the results in their
head, and so feel confident about what they're seeing. Not even I get a
warm feeling from a result that looks like 70330.345024097141 ...
Fred Drake [Mon, 21 May 2001 21:23:01 +0000 (21:23 +0000)]
Add a "See also" section with useful links. More should be added giving
pointers to information about the other mailbox formats; if anyone can
provide the information needed, please let me know!
Fred Drake [Mon, 21 May 2001 21:12:10 +0000 (21:12 +0000)]
Remove all files of expected output that contain only the name of the
test; there is no need to store this in a file if the actual test code
does not produce any output.
Fred Drake [Mon, 21 May 2001 21:08:12 +0000 (21:08 +0000)]
If the file containing expected output does not exist, assume that it
contains a single line of text giving the name of the output file. This
covers all tests that do not actually produce any output in the test code.
This patch changes the behaviour of the UTF-16 codec family. Only the
UTF-16 codec will now interpret and remove a *leading* BOM mark. Sub-
sequent BOM characters are no longer interpreted and removed.
UTF-16-LE and -BE pass through all BOM mark characters.
These changes should get the UTF-16 codec more in line with what
the Unicode FAQ recommends w/r to BOM marks.
Fred Drake [Mon, 21 May 2001 20:23:21 +0000 (20:23 +0000)]
Re-write the mailbox test suite to use PyUnit. Cover a lot more ground
for the Maildir mailbox format. This still does not address other mailbox
formats.
Guido van Rossum [Mon, 21 May 2001 20:17:17 +0000 (20:17 +0000)]
parse_declaration(): be more lenient in what we accept. We now
basically accept <!...> where the dots can be single- or double-quoted
strings or any other character except >.
Background: I found a real-life example that failed to parse with
the old assumption: http://www.opensource.org/licenses/jabberpl.html
contains a few constructs of the form <![if !supportLists]>...<![endif]>.
Barry Warsaw [Mon, 21 May 2001 19:58:23 +0000 (19:58 +0000)]
main(): default-domain argument to getopt.getopt() was missing a = to
indicate it took an argument. This closes SF patch #402223 by Bastian
Kleineidam.
Fred Drake [Sun, 20 May 2001 12:26:04 +0000 (12:26 +0000)]
Fix bug in smtplib example: the prompt said to end the message with ^D,
but doing so raised EOFError. This makes it work as advertised and
converts to string methods where reasonable.
Jack Jansen [Sat, 19 May 2001 12:34:59 +0000 (12:34 +0000)]
Include Carbon/Carbon.h in stead of universal headers, if appropriate.
Test for TARGET_API_MAC_OS8 in stead of !TARGET_API_MAC_CARBON where
appropriate.
Tim Peters [Sat, 19 May 2001 07:04:38 +0000 (07:04 +0000)]
Bugfix candidate.
Two exceedingly unlikely errors in dictresize():
1. The loop for finding the new size had an off-by-one error at the
end (could over-index the polys[] vector).
2. The polys[] vector ended with a 0, apparently intended as a sentinel
value but never used as such; i.e., it was never checked, so 0 could
have been used *as* a polynomial.
Neither bug could trigger unless a dict grew to 2**30 slots; since that
would consume at least 12GB of memory just to hold the dict pointers,
I'm betting it's not the cause of the bug Fred's tracking down <wink>.
Jeremy Hylton [Fri, 18 May 2001 20:57:38 +0000 (20:57 +0000)]
vgetargs1() and vgetargskeywords(): Replace uses of PyTuple_Size() and
PyTuple_GetItem() with PyTuple_GET_SIZE() and PyTuple_GET_ITEM().
The code has already done a PyTuple_Check().
Jeremy Hylton [Fri, 18 May 2001 20:53:14 +0000 (20:53 +0000)]
Add a second special case to the inline function call code in eval_code2().
If we have a PyCFunction (builtin) and it is METH_VARARGS only, load
the args and dispatch to call_cfunction() directly. This provides a
small speedup for perhaps the most common function calls -- builtins.
Fred Drake [Fri, 18 May 2001 15:32:59 +0000 (15:32 +0000)]
Added test suite for the new HTMLParser module, originally from the
TAL/PageTemplate package for Zope. This only needed a little boilerplate
change; the tests themselves are unchanged.
Guido van Rossum [Fri, 18 May 2001 14:50:52 +0000 (14:50 +0000)]
A much improved HTML parser -- a replacement for sgmllib. The API is
derived from but not quite compatible with that of sgmllib, so it's a
new file. I suppose it needs documentation, and htmllib needs to be
changed to use this instead of sgmllib, and sgmllib needs to be
declared obsolete. But that can all be done later.
This code was first published as part of TAL (part of Zope Page
Templates), but that was strongly based on sgmllib anyway. Authors
are Fred drake and Guido van Rossum.
Tim Peters [Thu, 17 May 2001 22:25:34 +0000 (22:25 +0000)]
Speed dictresize by collapsing its two passes into one; the reason given
in the comments for using two passes was bogus, as the only object that
can get decref'ed due to the copy is the dummy key, and decref'ing dummy
can't have side effects (for one thing, dummy is immortal! for another,
it's a string object, not a potentially dangerous user-defined object).