Tim Peters [Mon, 1 Apr 2002 19:23:44 +0000 (19:23 +0000)]
Restructured my pool-management overview in terms of the three
possible pool states. I think it's much clearer now.
Added a new long overdue block-management overview comment block.
I believe the comments are in good shape now.
Added two comments about possible small optimizations (one getting rid
of runtime multiplications at the cost of a new pool_header member; the
other getting rid of runtime divisions and the pool_header capacity
member, at the cost of a static const vector of 32 uints).
Jeremy Hylton [Mon, 1 Apr 2002 18:53:36 +0000 (18:53 +0000)]
Update documentation of code objects.
Split the description of co_flags into two paragraphs. The first
describes the flags that are used for non-future purposes, where
CO_GENERATOR was added. The second describes __future__'s use of
co_flags and mentions the only one currently meaningful,
CO_FUTURE_DIVISION.
Tim Peters [Mon, 1 Apr 2002 06:04:21 +0000 (06:04 +0000)]
New PYMALLOC_DEBUG function void _PyMalloc_DebugDumpStats(void).
This displays stats about the # of arenas, pools, blocks and bytes, to
stderr, both used and reserved but unused.
CAUTION: Because PYMALLOC_DEBUG is on, the debug malloc routine adds
16 bytes to each request. This makes each block appear two size classes
higher than it would be if PYMALLOC_DEBUG weren't on.
So far, playing with this confirms the obvious: there's a lot of activity
in the "small dict" size class, but nothing in the core makes any use of
the 8-byte or 16-byte classes.
Guido van Rossum [Sun, 31 Mar 2002 23:38:48 +0000 (23:38 +0000)]
Hopeful fix for SF bug 503031: urllib.py: open_http() host problem.
I really can't test this, but from reading the discussion in that bug
report, it's likely that this works. It may also close a whole bunch
of other bug reports related to urllib and proxies on Windows, but who
knows.
Jack Jansen [Sun, 31 Mar 2002 22:01:33 +0000 (22:01 +0000)]
Added a "run with commandline Python" flag. Works in MachoPython, should work
in OSX MacPython (untested), and should be disabled/removed in OS9 (but
that doesn't happen yet).
Neal Norwitz [Sun, 31 Mar 2002 15:27:00 +0000 (15:27 +0000)]
Remove METH_OLDARGS:
Convert METH_OLDARGS -> METH_VARARGS: also PyArg_Parse -> PyArg_ParseTuple
Convert METH_OLDARGS -> METH_NOARGS: remove args parameter
Please review. All tests pass, but some modules don't have tests.
I spot checked various functions to try to make sure nothing broke.
Tim Peters [Sun, 31 Mar 2002 02:18:01 +0000 (02:18 +0000)]
_PyMalloc_Free(): As was already done for _PyMalloc_Malloc, rearranged
the code so that the most frequent cases come first. Added comments.
Found a hidden assumption that a pool contains room for at least two
blocks, and added an assert to catch a violation if it ever happens in
a place where that matters. Gave the normal "I allocated this block"
case a longer basic block to work with before it has to do its first
branch (via breaking apart an embedded assignment in an "if", and
hoisting common code out of both branches).
Tim Peters [Sat, 30 Mar 2002 21:36:04 +0000 (21:36 +0000)]
It's once again thought safe to call the pymalloc free/realloc with an
address obtained from system malloc/realloc without holding the GIL.
When the vector of arena base addresses has to grow, the old vector is
deliberately leaked. This makes "stale" x-thread references safe.
arenas and narenas are also declared volatile, and changed in an order
that prevents a thread from picking up a value of narenas too large
for the value of arenas it sees.
Added more asserts.
Fixed an old inaccurate comment.
Added a comment explaining why it's safe to call pymalloc free/realloc
with an address obtained from system malloc/realloc even when arenas is
still NULL (this is obscure, since the ADDRESS_IN_RANGE macro
appears <wink> to index into arenas).
Tim Peters [Sat, 30 Mar 2002 07:04:41 +0000 (07:04 +0000)]
Now that we're no longer linking arenas together, there's no need to
waste the first pool if malloc happens to return a pool-aligned address.
This means the number of pools per arena can now vary by 1. Unfortunately,
the code counted up from 0 to a presumed constant number of pools. So
changed the increasing "watermark" counter to a decreasing "nfreepools"
counter instead, and fiddled various stuff accordingly. This also allowed
getting rid of two more macros.
Also changed the code to align the first address to a pool boundary
instead of a page boundary. These are two parallel sets of macro #defines
that happen to be identical now, but the page macros are in theory more
restrictive (bigger), and there's simply no reason I can see that it
wasn't aligning to the less restrictive pool size all along (the code
only relies on pool alignment).
Hmm. The "page size" macros aren't used for anything *except* defining
the pool size macros, and the comments claim the latter isn't necessary.
So this has the feel of a layer of indirection that doesn't serve a
purpose; should probably get rid of the page macros now.
Tim Peters [Sat, 30 Mar 2002 06:20:23 +0000 (06:20 +0000)]
Retract the claim that this is always safe if PyMem_{Del, DEL, Free, FREE}
are called without the GIL. It's incredibly unlikely to fail, but I can't
make this bulletproof without either adding a lock for exclusion, or
giving up on growing the arena base-address vector (it would be safe if
this were a static array).
Tim Peters [Sat, 30 Mar 2002 06:09:22 +0000 (06:09 +0000)]
Lots of changes:
+ A new scheme for determining whether an address belongs to a pymalloc
arena. This should be 100% reliable. The poolp->pooladdr and
poolp->magic members are gone. A new poolp->arenaindex member takes
their place. Note that the pool header overhead doesn't actually
shrink, though, since the header is padded to a multiple of 8 bytes.
+ _PyMalloc_Free and _PyMalloc_Realloc should now be safe to call for
any legit address, whether obtained from a _PyMalloc function or from
the system malloc/realloc. It should even be safe to call
_PyMalloc_Free when *not* holding the GIL, provided that the passed-in
address was obtained from system malloc/realloc. Since this is
accomplished without any locks, you better believe the code is subtle.
I hope it's sufficiently commented.
+ The above implies we don't need the new PyMalloc_{New, NewVar, Del}
API anymore, and could switch back to PyObject_XXX without breaking
existing code mixing PyObject_XXX with PyMem_{Del, DEL, Free, FREE}.
Nothing is done here about that yet, and I'd like to see this new
code exercised more first.
+ The small object threshhold is boosted to 256 (the max). We should
play with that some more, but the old 64 was way too small for 2.3.
+ Getting a new arena is now done via new function new_arena().
+ Removed some unused macros, and squashed out some macros that were
used only once to define other macros.
+ Arenas are no longer linked together. A new vector of arena base
addresses had to be created anyway to make address classification
bulletproof.
+ A lot of the patch size is an illusion: given the way address
classification works now, it was more convenient to switch the
sense of the prime "if" tests in the realloc and free functions,
so the "if" and "else" blocks got swapped.
+ Assorted minor code, comment and whitespace cleanup.
Fred Drake [Fri, 29 Mar 2002 22:45:28 +0000 (22:45 +0000)]
Started updating information about defining attributes on types.
There's still a long way to go, but we're starting to see some real
content in the docs.
Jack Jansen [Fri, 29 Mar 2002 21:26:04 +0000 (21:26 +0000)]
Completely revamped newline handling. PyEdit is now newline-preserving
(if a single newline convention is used in the sourcefile), and the
"save options" has a newline style radio button.
The creator radio button also has the new choices PythonW and None.
Jack Jansen [Fri, 29 Mar 2002 21:23:47 +0000 (21:23 +0000)]
Allow file without filetype as long as they end in ".py".
Added a -D flag (can really only be specified on OSX commandline) to not
revector sys.stderr, for debugging the IDE itself. Not sure whether
this should stay.
Fred Drake [Thu, 28 Mar 2002 23:12:09 +0000 (23:12 +0000)]
Move some of the longer example code to external fragments, and
include them using \verbatiminput. This has the advantage that pages
can still break at reasonable places, and examples that go longer than
a page won't get cut off.
Make a few small markup adjustments for consistency.
Explain that PyObject_New() is not a C function but a polymorphic
beast that returns a pointer to the type that's passed as the first
arg.
Explain why type objects use the PyObject_VAR_HEAD.
Jeremy Hylton [Thu, 28 Mar 2002 23:01:56 +0000 (23:01 +0000)]
Fix getcomments() so that it doesn't fail with TypeErrors.
It appears that getcomments() can get called for classes defined in
C. Since these don't have source code, it can't do anything useful.
A function buried many levels deep was raising a TypeError that was
not caught.