Victor Stinner [Tue, 27 Jan 2015 23:30:40 +0000 (00:30 +0100)]
asyncio: sync with Tulip
* Remove unused SSLProtocol._closing attribute
* test_sslproto: skip test if ssl module is missing
* Python issue #23208: Don't use the traceback of the current handle if we
already know the traceback of the source. The handle may be more revelant,
but having 3 tracebacks (handle, source, exception) becomes more difficult to
read. The handle may be preferred later but it requires more work to make
this choice.
Victor Stinner [Mon, 26 Jan 2015 21:30:49 +0000 (22:30 +0100)]
asyncio, Tulip issue 204: Fix IocpProactor.recv()
If ReadFile() fails with ERROR_BROKEN_PIPE, the operation is not pending: don't
register the overlapped.
I don't know if WSARecv() can fail with ERROR_BROKEN_PIPE. Since
Overlapped.WSARecv() already handled ERROR_BROKEN_PIPE, let me guess that it
has the same behaviour than ReadFile().
If UnregisterWaitEx() fais with ERROR_IO_PENDING, it doesn't mean that the wait
is unregistered yet. We still have to wait until the wait is cancelled.
In debug mode, BaseEventLoop._run_once() now sets the
BaseEventLoop._current_handle attribute to the handle currently executed.
In release mode or when no handle is executed, the attribute is None.
BaseEventLoop.default_exception_handler() displays the traceback of the current
handle if available.
Gregory P. Smith [Fri, 23 Jan 2015 06:55:00 +0000 (22:55 -0800)]
revert 7b833bd1f509. I misread the side effect that the code was triggering.
*any* kwarg supplied to _assert_python causes it to not append -E to the
command line flags so without='-E' does effectively work.
Gregory P. Smith [Fri, 23 Jan 2015 06:04:16 +0000 (22:04 -0800)]
Remove the unimplemented but ignored without='-E' parameters being passed to
script_helper.assert_python_failure(). No such feature has ever existed,
thus it doesn't do what the comment claims. (It does add a 'without'
variable to the environment of the child process but that was not intended)
Gregory P. Smith [Fri, 23 Jan 2015 01:33:28 +0000 (17:33 -0800)]
Only pass -E to the child interpreter if our interpreter was running in that
mode. Explicitly remove the PYTHONFAULTHANDLER environment variable before
launching a child interpreter when its presence would impact the test (the
reason -E was being used in the first place).
This enables running the test in an environment where other Python environment
variables must be set in order for things to run (such as using PYTHONHOME to
tell an embedded interpreter where it should think it lives).
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.
This change removes multiple hacks in IocpProactor.
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.
Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.
_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().
To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.
The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.
Other changes:
* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
result if the wait already completed. I'm not sure that it's safe to
call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
signaled, but may be signaled for the next loop iteration. It avoids to
block forever IocpProactor.close() if a wait was cancelled, and it may also
avoid some "... unexpected event ..." warnings.
Ned Deily [Sun, 18 Jan 2015 00:57:19 +0000 (16:57 -0800)]
Issue #23211: Workaround test_logging failure on some OS X 10.6 systems:
getaddrinfo("localhost") can fail depending on the name server configuration,
use "127.0.0.0" instead.
Victor Stinner [Thu, 15 Jan 2015 13:24:55 +0000 (14:24 +0100)]
asyncio: Close transports in tests
* Use test_utils.run_briefly() to execute pending calls to really close
transports
* sslproto: mock also _SSLPipe.shutdown(), it's need to close the transport
* pipe test: the test doesn't close explicitly the PipeHandle, so ignore
the warning instead
* test_popen: use the context manager ("with p:") to explicitly close pipes
Don't call immediatly self._process_write_backlog() but schedule the call using
call_soon(). _on_handshake_complete() can be called indirectly from
_process_write_backlog(), and _process_write_backlog() is not reentrant.
Victor Stinner [Wed, 14 Jan 2015 23:04:21 +0000 (00:04 +0100)]
asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
(BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
state in the __repr__() method of _UnixReadPipeTransport and
_UnixWritePipeTransport classes.
Victor Stinner [Wed, 14 Jan 2015 01:10:33 +0000 (02:10 +0100)]
Python issue #23173: sync with Tulip
* If an exception is raised during the creation of a subprocess, kill the
subprocess (close pipes, kill and read the return status). Log an error in
such case.
* Fix SubprocessStreamProtocol.connection_made() to handle cancelled waiter.
Add unit test cancelling subprocess methods.
Victor Stinner [Tue, 13 Jan 2015 23:53:37 +0000 (00:53 +0100)]
Issue #23198: Reactor asyncio.StreamReader
- Add a new _wakeup_waiter() method
- Replace _create_waiter() method with a _wait_for_data() coroutine function
- Use the value None instead of True or False to wake up the waiter
Victor Stinner [Tue, 13 Jan 2015 23:19:09 +0000 (00:19 +0100)]
Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.
The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).
Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.
This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!
It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
Victor Stinner [Tue, 13 Jan 2015 15:11:19 +0000 (16:11 +0100)]
Issue #22922: Fix ProactorEventLoop.close()
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
Victor Stinner [Tue, 13 Jan 2015 08:58:33 +0000 (09:58 +0100)]
Issue #23209, #23225: selectors.BaseSelector.close() now clears its internal
reference to the selector mapping to break a reference cycle. Initial patch
written by Martin Richard.
Victor Stinner [Fri, 9 Jan 2015 20:32:05 +0000 (21:32 +0100)]
asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
loop parameter is not set. Add unit tests to ensure that constructor of
StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
Victor Stinner [Fri, 9 Jan 2015 00:42:52 +0000 (01:42 +0100)]
asyncio: sync with Tulip
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
socket.