Tim Peters [Fri, 26 May 2006 14:02:05 +0000 (14:02 +0000)]
Explicitly close files. I'm trying to stop the frequent spurious test_tarfile
failures on Windows buildbots, but it's hard to know how since the regrtest
failure output is useless here, and it never fails when a buildbot slave runs
test_tarfile the second time in verbose mode.
Andrew Dalke [Fri, 26 May 2006 14:00:45 +0000 (14:00 +0000)]
Changes to string.split/rsplit on whitespace to preallocate space in the
results list.
Originally it allocated 0 items and used the list growth during append. Now
it preallocates 12 items so the first few appends don't need list reallocs.
("Here are some words ."*2).split(None, 1) is 7% faster
("Here are some words ."*2).split() is is 15% faster
(Your milage may vary, see dealership for details.)
File parsing like this
for line in f:
count += len(line.split())
is also about 15% faster. There is a slowdown of about 3% for large
strings because of the additional overhead of checking if the append is
to a preallocated region of the list or not. This will be the rare case.
It could be improved with special case code but we decided it was not
useful enough.
There is a cost of 12*sizeof(PyObject *) bytes per list. For the normal
case of file parsing this is not a problem because of the lists have
a short lifetime. We have not come up with cases where this is a problem
in real life.
I chose 12 because human text averages about 11 words per line in books,
one of my data sets averages 6.2 words with a final peak at 11 words per
line, and I work with a tab delimited data set with 8 tabs per line (or
9 words per line). 12 encompasses all of these.
Also changed the last rstrip code to append then reverse, rather than
doing insert(0). The strip() and rstrip() times are now comparable.
Ronald Oussoren [Fri, 26 May 2006 12:23:20 +0000 (12:23 +0000)]
- Remove previous version of the binary distribution script for OSX
- Some small bugfixes for the IDLE.app wrapper
- Tweaks to build-installer to ensure that python gets build in the right way,
including sqlite3.
- Updated readme files
Ronald Oussoren [Fri, 26 May 2006 11:43:26 +0000 (11:43 +0000)]
Integrate installing a framework in the 'make install'
target. Until now users had to use 'make frameworkinstall'
to install python when it is configured with '--enable-framework'.
This tends to confuse users that don't hunt for readme files
hidden in platform specific directories :-)
Ronald Oussoren [Fri, 26 May 2006 11:38:39 +0000 (11:38 +0000)]
- Search the sqlite specific search directories
after the normal include directories when looking
for the version of sqlite to use.
- On OSX:
* Extract additional include and link directories
from the CFLAGS and LDFLAGS, if the user has
bothered to specify them we might as wel use them.
* Add '-Wl,-search_paths_first' to the extra_link_args
for readline and sqlite. This makes it possible to
use a static library to override the system provided
dynamic library.
Fredrik Lundh [Fri, 26 May 2006 11:29:39 +0000 (11:29 +0000)]
needforspeed: added Py_LOCAL macro, based on the LOCAL macro used
for SRE and others. applied Py_LOCAL to relevant portion of ceval,
which gives a 1-2% speedup on my machine. ymmv.
Fredrik Lundh [Fri, 26 May 2006 09:46:59 +0000 (09:46 +0000)]
needforspeed: use METH_O for argument handling, which made partition some
~15% faster for the current tests (which is noticable faster than a corre-
sponding find call). thanks to neal-who-never-sleeps for the tip.
The SIGCHECK macro defined here has always been bizarre, but
it apparently causes compiler warnings on "Sun Studio 11".
I believe the warnings are bogus, but it doesn't hurt to make
the macro definition saner.
Brett Cannon [Thu, 25 May 2006 21:33:11 +0000 (21:33 +0000)]
Change test_values so that it compares the lowercasing of group names since getgrall() can return all lowercase names while getgrgid() returns proper casing.
Bob Ippolito [Thu, 25 May 2006 18:44:50 +0000 (18:44 +0000)]
Struct now unpacks to PY_LONG_LONG directly when possible, also include #ifdef'ed out code that will return int instead of long when in bounds (not active since it's an API and doc change)
Andrew Dalke [Thu, 25 May 2006 18:18:39 +0000 (18:18 +0000)]
Added overflow test for adding two (very) large strings where the
new string is over max Py_ssize_t. I have no way to test it on my
box or any box I have access to. At least it doesn't break anything.
Andrew Dalke [Thu, 25 May 2006 17:53:00 +0000 (17:53 +0000)]
Fixed problem identified by Georg. The special-case in-place code for replace
made a copy of the string using PyString_FromStringAndSize(s, n) and modify
the copied string in-place. However, 1 (and 0) character strings are shared
from a cache. This cause "A".replace("A", "a") to change the cached version
of "A" -- used by everyone.
Now may the copy with NULL as the string and do the memcpy manually. I've
added regression tests to check if this happens in the future. Perhaps
there should be a PyString_Copy for this case?
Tim Peters [Thu, 25 May 2006 17:34:03 +0000 (17:34 +0000)]
A new table to help string->integer conversion was added yesterday to
both mystrtoul.c and longobject.c. Share the table instead. Also
cut its size by 64 entries (they had been used for an inscrutable
trick originally, but the code no longer tries to use that trick).
Added a new macro, Py_IS_FINITE(X). On windows there is an intrinsic for this and it is more efficient than to use !Py_IS_INFINITE(X) && !Py_IS_NAN(X). No change on other platforms
Fredrik Lundh [Thu, 25 May 2006 15:49:45 +0000 (15:49 +0000)]
needforspeed: make new upper/lower work properly for single-character
strings too... (thanks to georg brandl for spotting the exact problem
faster than anyone else)
Tim Peters [Wed, 24 May 2006 21:10:40 +0000 (21:10 +0000)]
Heavily fiddled variant of patch #1442927: PyLong_FromString optimization.
``long(str, base)`` is now up to 6x faster for non-power-of-2 bases. The
largest speedup is for inputs with about 1000 decimal digits. Conversion
from non-power-of-2 bases remains quadratic-time in the number of input
digits (it was and remains linear-time for bases 2, 4, 8, 16 and 32).
Speedups at various lengths for decimal inputs, comparing 2.4.3 with
current trunk. Note that it's actually a bit slower for 1-digit strings:
Andrew Dalke [Wed, 24 May 2006 18:55:37 +0000 (18:55 +0000)]
Added a slew of test for string replace, based various corner cases from
the Need For Speed sprint coding. Includes commented out overflow tests
which will be uncommented once the code is fixed.
This test will break the 8-bit string tests because
"".replace("", "A") == "" when it should == "A"
We have a fix for it, which should be added tomorrow.
Fredrik Lundh [Wed, 24 May 2006 14:28:11 +0000 (14:28 +0000)]
needforspeed: use "fastsearch" for count and findstring helpers. this
results in a 2.5x speedup on the stringbench count tests, and a 20x (!)
speedup on the stringbench search/find/contains test, compared to 2.5a2.
for more on the algorithm, see:
http://effbot.org/zone/stringlib.htm
if you get weird results, you can disable the new algoritm by undefining
USE_FAST in Objects/unicodeobject.c.
Tim Peters [Tue, 23 May 2006 21:51:35 +0000 (21:51 +0000)]
test_struct grew weird behavior under regrtest.py -R,
due to a module-level cache. Clearing the cache should
make it stop showing up in refleak reports.