Tim Peters [Mon, 8 Oct 2001 06:13:19 +0000 (06:13 +0000)]
Widespread random code cleanup.
Most of this code was old enough to vote. Examples of cleanups:
+ Backslashes were used for line continuation even inside unclosed
bracket structures, from back in the days that was still needed.
+ There was no use of % formats, and e.g. the old fpformat module was
still used to format floats "by hand" in conjunction with rjust().
+ There was even use of a do-nothing .ignore() method to tack on to the
end of a chain of method calls, else way back when Python would print
the non-None result (as it does now in an interactive session -- it
*used* to do that in batch mode too).
+ Perhaps controversial (although I can't imagine why for real <wink>),
used augmented assignment where helpful. Stuff like
Implement isinstance(x, (A, B, ...)). Note that we only allow tuples,
not other sequences (then we'd have to except strings, and we'd still
be susceptible to recursive attacks).
Tim Peters [Sun, 7 Oct 2001 08:35:44 +0000 (08:35 +0000)]
Guido points out that the comments for self.cur[2] were subtly but
seriously wrong. This started out by just fixing the docs, but then it
occurred to me that the doc confusion propagated into misleading vrbl names
too, so I also renamed those to match reality. As a result, INO the time
computations are much easier to understand now (within the limitations of
vast quantities of 3-character names <wink>).
Tim Peters [Sun, 7 Oct 2001 03:54:51 +0000 (03:54 +0000)]
Guido suggests, and I agree, to insist that SIZEOF_VOID_P be a power of 2.
This simplifies the rounding in _PyObject_VAR_SIZE, allows to restore the
pre-rounding calling sequence, and allows some nice little simplifications
in its callers. I'm still making it return a size_t, though.
Tim Peters [Sun, 7 Oct 2001 03:12:08 +0000 (03:12 +0000)]
Remove code and docs for the OldProfile and HotProfile classes: code
hasn't worked in years, docs were wrong, and they aren't interesting
anymore regardless.
Tim Peters [Sat, 6 Oct 2001 21:27:34 +0000 (21:27 +0000)]
_PyObject_VAR_SIZE: always round up to a multiple-of-pointer-size value.
As Guido suggested, this makes the new subclassing code substantially
simpler. But the mechanics of doing it w/ C macro semantics are a mess,
and _PyObject_VAR_SIZE has a new calling sequence now.
Question: The PyObject_NEW_VAR macro appears to be part of the public API.
Regardless of what it expands to, the notion that it has to round up the
memory it allocates is new, and extensions containing the old
PyObject_NEW_VAR macro expansion (which was embedded in the
PyObject_NEW_VAR expansion) won't do this rounding. But the rounding
isn't actually *needed* except for new-style instances with dict pointers
after a variable-length blob of embedded data. So my guess is that we do
not need to bump the API version for this (as the rounding isn't needed
for anything an extension can do unless it's recompiled anyway). What's
your guess?
Tim Peters [Sat, 6 Oct 2001 19:04:01 +0000 (19:04 +0000)]
Repaired the debug Windows deaths in test_descr, by allocating enough
pad memory to properly align the __dict__ pointer in all cases.
gcmodule.c/objimpl.h, _PyObject_GC_Malloc:
+ Added a "padding" argument so that this flavor of malloc can allocate
enough bytes for alignment padding (it can't know this is needed, but
its callers do).
typeobject.c, PyType_GenericAlloc:
+ Allocated enough bytes to align the __dict__ pointer.
+ Sped and simplified the round-up-to-PTRSIZE logic.
+ Added blank lines so I could parse the if/else blocks <0.7 wink>.
Tim Peters [Sat, 6 Oct 2001 17:45:17 +0000 (17:45 +0000)]
_PyObject_GetDictPtr():
+ Use the _PyObject_VAR_SIZE macro to compute object size.
+ Break the computation into lines convenient for debugger inspection.
+ Speed the round-up-to-pointer-size computation.
Tim Peters [Sat, 6 Oct 2001 08:03:20 +0000 (08:03 +0000)]
_PyObject_GC_Malloc(): split a complicated line in two. As is, there was
no way to talk the debugger into showing me how many bytes were being
allocated.
Tim Peters [Fri, 5 Oct 2001 23:15:10 +0000 (23:15 +0000)]
The fix to profile semantics broke the miserable but advertised way to
derive Profile subclasses. This patch repairs that, restoring
negative tuple indices. Yuck? You bet.
Fred Drake [Fri, 5 Oct 2001 22:00:24 +0000 (22:00 +0000)]
Adjust the _weakref module to use the public API for the weak reference
objects. This is now simply a shim to give weakref.py access to the
underlying implementation.
Fred Drake [Fri, 5 Oct 2001 21:50:08 +0000 (21:50 +0000)]
weakref.ReferenceError becomes a built-in exception now that weak ref objects
are moving into the core; with these changes, it will be possible for the
exception to be raised without the weakref module ever being imported.
Enable GC for new-style instances. This touches lots of files, since
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self). It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types. Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again. I'm not
sure if doing something like
void xxx_dealloc(PyObject *self)
{
if (PyXxxCheckExact(self))
PyObject_DEL(self);
else
self->ob_type->tp_free(self);
}
is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
Tim Peters [Fri, 5 Oct 2001 20:06:47 +0000 (20:06 +0000)]
A regexp example was rendered as
foo\d
when it was clearly intended to render as
foo$
Fred, is this a right way to fix it? If not, the earlier place in the
same paragraph that does render as
foo$
is also wrong.
Apply modified SF patch 467580: ConfigParser.getboolean(): FALSE, TRUE.
This patch allows ConfigParser.getboolean() to interpret TRUE,
FALSE, YES, NO, ON and OFF instead just '0' and '1'.
While just allowing '0' and '1' sounds more correct users often
demand to use more descriptive directives in configuration
files. Instead of forcing every programmer do brew his own
solution a system should include the batteries for this.
[My modification to the patch is a slight rewording of the docstring
and use of lowercase instead of uppercase templates. The code is
still case sensitive. GvR.]
Fred Drake [Thu, 4 Oct 2001 19:46:07 +0000 (19:46 +0000)]
run_suite(): If testclass is not available, provide an even more general
error message.
run_unittest(): Provide the testclass to run_suite() so it can construct
the error message.
This closes SF bug #467763.
Tim Peters [Thu, 4 Oct 2001 19:44:10 +0000 (19:44 +0000)]
Changed the reindenter to strip only trailing spaces and tabs from lines,
not other control characters string.rstrip() got rid of. This caters to
the \f thingies Barry likes putting in Python source files.
Fred Drake [Thu, 4 Oct 2001 19:26:43 +0000 (19:26 +0000)]
Fix bug in profiler modifications detected only in debug builds.
The new profiler event stream includes a "return" event even when an
exception is being propogated, but the machinery that called the profile
hook did not save & restore the exception. In debug mode, the exception
was detected during the execution of the profile callback, which did not
have the proper internal flags set for the exception. Saving & restoring
the exception state solves the problem.
Barry Warsaw [Thu, 4 Oct 2001 16:27:04 +0000 (16:27 +0000)]
Script arguments localhost:localport and remotehost:remoteport are now
optional, and default to `localhost' and ports 8025 and 25
respectively.
SMTPChannel.__init__(): Calculate __fqdn using socket.getfqdn()
instead of gethostby*() and friends. This allows us to run this
script even if we don't have access to dns (assuming the localhost is
configured properly).
Fred Drake [Thu, 4 Oct 2001 14:48:42 +0000 (14:48 +0000)]
Rationalize the events passed to the profiler (no changes for the tracer).
The profiler does not need to know anything about the exception state,
so we no longer call it when an exception is raised. We do, however,
make sure we *always* call the profiler when we exit a frame. This
ensures that timing events are more easily isolated by a profiler and
finally clauses that do a lot of work don't have their time
mis-allocated.
When an exception is propogated out of the frame, the C callback for
the profiler now receives a PyTrace_RETURN event with an arg of NULL;
the Python-level profile hook function will see a 'return' event with
an arg of None. This means that from Python it is impossible for the
profiler to determine if the frame exited with an exception or if it
returned None, but this doesn't matter for profiling. A C-based
profiler could tell the difference, but this doesn't seem important.
ceval.c:eval_frame(): Simplify the code in two places so that the
profiler is called for every exit from a frame
and not for exceptions.
sysmodule.c:profile_trampoline(): Make sure we don't expose Python
code to NULL; use None instead.
Tim Peters [Thu, 4 Oct 2001 05:27:00 +0000 (05:27 +0000)]
SF bug [#467331] ClassType.__doc__ always None.
For a dynamically constructed type object, fill in the tp_doc slot with
a copy of the argument dict's "__doc__" value, provided the latter exists
and is a string.
NOTE: I don't know what to do if it's a Unicode string, so in that case
tp_doc is left NULL (which shows up as Py_None if you do Class.__doc__).
Note that tp_doc holds a char*, not a general PyObject*.
Hopefully fix the profiler right. Add a test suite that checks that
it deals correctly with some anomalous cases; according to this test
suite I've fixed it right.
The anomalous cases had to do with 'exception' events: these aren't
generated when they would be most helpful, and the profiler has to
work hard to recover the right information. The problems occur when C
code (such as hasattr(), which is used as the example here) calls back
into Python code and clears an exception raised by that Python code.
Consider this example:
def foo():
hasattr(obj, "bar")
Where obj is an instance from a class like this:
class C:
def __getattr__(self, name):
raise AttributeError
The profiler sees the following sequence of events:
call (foo)
call (__getattr__)
exception (in __getattr__)
return (from foo)
Previously, the profiler would assume the return event returned from
__getattr__. An if statement checking for this condition and raising
an exception was commented out... This version does the right thing.
dynamics(): add a dummy __getattr__ method to the C class so that the
test for modifying __getattr__ works, now that slot_tp_getattr_hook
zaps the slot if there's no hook. Added an XXX comment with a ref
back to slot_tp_getattr_hook.
typeobject.c, slot_tp_gettattr_hook(): fix the speedup hack -- the
test for getattribute==NULL was bogus because it always found
object.__getattribute__. Pick it apart using the trick we learned
from slot_sq_item, and if it's just a wrapper around
PyObject_GenericGetAttr, zap it. Also added a long XXX comment
explaining the consequences.
*EXPERIMENTAL* speedup of slot_sq_item. This sped up the following
test dramatically:
class T(tuple): __dynamic__ = 1
t = T(range(1000))
for i in range(1000): tt = tuple(t)
The speedup was about 5x compared to the previous state of CVS (1.7
vs. 8.8, in arbitrary time units). But it's still more than twice as
slow as as the same test with __dynamic__ = 0 (0.8).
I'm not sure that I really want to go through the trouble of this kind
of speedup for every slot. Even doing it just for the most popular
slots will be a major effort (the new slot_sq_item is 40+ lines, while
the old one was one line with a powerful macro -- unfortunately the
speedup comes from expanding the macro and doing things in a way
specific to the slot signature).
An alternative that I'm currently considering is sketched in PLAN.txt:
trap setattr on type objects. But this will require keeping track of
all derived types using weak references.
Tim Peters [Wed, 3 Oct 2001 04:08:26 +0000 (04:08 +0000)]
SF bug [#467336] doctest failures w/ new-style classes.
Taught doctest about static methods, class methods, and property docstrings
in new-style classes. As for inspect.py/pydoc.py before it, the new stuff
needed didn't really fit into the old architecture (but was less of a
strain to force-fit here).
New-style class docstrings still aren't found, but that's the subject
of a different bug and I want to fix that right instead of hacking around
it in doctest.
Mark treatment of binary operators for __rop__-before-__op__ as done.
Add more detail about the speed optimizations needed for __dynamic__.
The weak reference solution becomes more attractive...
call_method(), call_maybe(): fix a performance bug: the argument
pointing to a static variable to hold the object form of the string
was never used, causing endless calls to PyString_InternFromString().
One particular test (with lots of __getitem__ calls) became a third
faster with this!
Tim Peters [Tue, 2 Oct 2001 22:47:08 +0000 (22:47 +0000)]
SF patch [#466616] Exclude imported items from doctest.
Another installment; the new functionality wasn't actually enabled in
normal use, only in the strained use checked by the test case.
Tim Peters [Tue, 2 Oct 2001 21:32:07 +0000 (21:32 +0000)]
SF bug [#467265] Compile errors on SuSe Linux on IBM/s390.
Unknown whether this fixes it.
- stringobject.c, PyString_FromFormatV: don't assume that va_list is of
a type that can be copied via an initializer.
- errors.c, PyErr_Format: add a va_end() to balance the va_start().
Add Garbage Collection support to new-style classes (not yet to their
instances).
Also added GC support to various auxiliary types: super, property,
descriptors, wrappers, dictproxy. (Only type objects have a tp_clear
field; the other types are.)
One change was necessary to the GC infrastructure. We have statically
allocated type objects that don't have a GC header (and can't easily
be given one) and heap-allocated type objects that do have a GC
header. Giving these different metatypes would be really ugly: I
tried, and I had to modify pickle.py, cPickle.c, copy.py, add a new
invent a new name for the new metatype and make it a built-in, change
affected tests... In short, a mess. So instead, we add a new type
slot tp_is_gc, which is a simple Boolean function that determines
whether a particular instance has GC headers or not. This slot is
only relevant for types that have the (new) GC flag bit set. If the
tp_is_gc slot is NULL (by far the most common case), all instances of
the type are deemed to have GC headers. This slot is called by the
PyObject_IS_GC() macro (which is only used twice, both times in
gcmodule.c).
I also changed the extern declarations for a bunch of GC-related
functions (_PyObject_GC_Del etc.): these always exist but objimpl.h
only declared them when WITH_CYCLE_GC was defined, but I needed to be
able to reference them without #ifdefs. (When WITH_CYCLE_GC is not
defined, they do the same as their non-GC counterparts anyway.)
- The test for deepcopy() in pickles() was indented wrongly, so it got
run twice (one for binary pickle mode, one for text pickle mode; but
the test doesn't depend on the pickle mode).
- In verbose mode, show which subtest (pickle/cPickle/deepcopy, text/bin).
The error reporting here was a bit sparse. In verbose mode, the code
in run_test() referenced two non-existent variables, and in
non-verbose mode, the tests didn't report the actual number, when it
differed from the expected number. Fixed this.
Also added an extra call to gc.collect() at the start of test_all().
This will be needed when I check in the changes to add GC to new-style
classes.
Under certain conditions (sometimes triggered by the test suite),
"from xml.parsers import expat" succeeds but the imported expat module
is an empty shell. Make sure we don't be fooled by that.
Correct the URL for the license (only used when the LICENSE[.txt] file
is not found). Being fancy: insert the first 3 characters of
sys.version in the URL.
Tim Peters [Tue, 2 Oct 2001 03:53:41 +0000 (03:53 +0000)]
SF patch [#466616] Exclude imported items from doctest,
from Tim Hochberg. Also mucho fiddling to change the way doctest
determines whether a thing is a function, module or class. Under 2.2,
this really requires the functions in inspect.py (e.g., types.ClassType
is close to meaningless now, if not outright misleading).