../http.c:589:6: warning: logical not is only applied to the left hand side of this comparison [-Wlogical-not-parentheses]
if (!req->kind == EVHTTP_REQUEST || !REQ_VERSION_ATLEAST(req, 1, 1))
^ ~~
If the length of ```base_name``` is 0, then line 3125 reads 1 byte before the buffer. This will trigger a crash on ASAN-protected builds.
To reproduce:
Build libevent with ASAN:
```
$ CFLAGS='-fomit-frame-pointer -fsanitize=address' ./configure && make -j4
```
Put the attached ```resolv.conf``` and ```poc.c``` in the source directory and then do:
```
$ gcc -fsanitize=address -fomit-frame-pointer poc.c .libs/libevent.a
$ ./a.out
=================================================================
==22201== ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60060000efdf at pc 0x4429da bp 0x7ffe1ed47300 sp 0x7ffe1ed472f8
READ of size 1 at 0x60060000efdf thread T0
```
P.S. we can add a check earlier, but since this is very uncommon, I didn't add it.
Azat Khuzhin [Thu, 24 Mar 2016 09:49:47 +0000 (12:49 +0300)]
evdns: avoid double-free in evdns_base_free() for probing requests
http/cancel_by_host_no_ns:
OK ../test/regress_http.c:1384: assert(regress_dnsserver(data->base, &portnum, search_table))
OK ../test/regress_http.c:1387: assert(dns_base)
OK ../test/regress_http.c:1423: assert(evcon)
OK ../test/regress_http.c:1444: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/delay") != -1): 0 vs -1
OK ../test/regress_http.c:1455: assert(test_ok == 2): 2 vs 2
OK ../test/regress_http.c:1480: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/test") != -1): 0 vs -1[msg] Nameserver 127.0.0.1:55948 has failed: request timed out.
[msg] All nameservers have failed
OK ../test/regress_http.c:1274: assert(!req)
OK ../test/regress_http.c:1505: assert(evhttp_make_request(evcon, req, EVHTTP_REQ_GET, "/test") != -1): 0 vs -1
OK ../test/regress_http.c:1274: assert(!req)==19199== Invalid read of size 8
==19199== at 0x4CC285: evdns_cancel_request (evdns.c:2849)
==19199== by 0x4CEDB2: evdns_nameserver_free (evdns.c:4018)
==19199== by 0x4CEF5B: evdns_base_free_and_unlock (evdns.c:4052)
==19199== by 0x4CF13B: evdns_base_free (evdns.c:4088)
==19199== by 0x4617A3: http_cancel_test (regress_http.c:1518)
==19199== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199== by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199== by 0x491699: tinytest_main (tinytest.c:434)
==19199== by 0x47E0E0: main (regress_main.c:461)
==19199== Address 0x61e56d0 is 0 bytes inside a block of size 48 free'd
==19199== at 0x4C2AE6B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19199== by 0x4AAFFF: event_mm_free_ (event.c:3516)
==19199== by 0x4C5ADD: request_finished (evdns.c:693)
==19199== by 0x4CEE95: evdns_base_free_and_unlock (evdns.c:4040)
==19199== by 0x4CF13B: evdns_base_free (evdns.c:4088)
==19199== by 0x4617A3: http_cancel_test (regress_http.c:1518)
==19199== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199== by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199== by 0x491699: tinytest_main (tinytest.c:434)
==19199== by 0x47E0E0: main (regress_main.c:461)
==19199== Block was alloc'd at
==19199== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19199== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==19199== by 0x4CAAA2: nameserver_send_probe (evdns.c:2327)
==19199== by 0x4C50FF: nameserver_prod_callback (evdns.c:494)
==19199== by 0x4A564C: event_process_active_single_queue (event.c:1646)
==19199== by 0x4A5B95: event_process_active (event.c:1738)
==19199== by 0x4A6296: event_base_loop (event.c:1961)
==19199== by 0x4A5C1D: event_base_dispatch (event.c:1772)
==19199== by 0x46172C: http_cancel_test (regress_http.c:1507)
==19199== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==19199== by 0x490D5A: testcase_run_one (tinytest.c:252)
==19199== by 0x491699: tinytest_main (tinytest.c:434)
==19199==
Azat Khuzhin [Thu, 24 Mar 2016 11:05:28 +0000 (14:05 +0300)]
Merge branch 'bufev-cancellations-v5'
This patch set fixes bufferevent via http request cancellations (connect() and
dns-request), it survives tests and cancel.. with --no-fork, so this must be ok
(though I have one patch for dns layer pending).
But I'm not sure about cancel.. unit tests on win32, will fix disable them
later if they will differs (plus maybe we must make them skip-by-default?).
Fixes: #333
* bufev-cancellations-v5:
http: set fd to -1 unconditioally, to avoid leaking of DNS requests
test/http: cover NS timed out during request cancellations separatelly
http: avoid leaking of fd in evhttp_connection_free()
http: get fd from be layer during connection reset
be_sock: cancel in-progress dns requests
evdns: export cancel via callbacks in util (like async lib core/extra issues)
test/http: request cancellation with resolving/{conn,write}-timeouts in progress
Azat Khuzhin [Thu, 24 Mar 2016 10:38:05 +0000 (13:38 +0300)]
http: set fd to -1 unconditioally, to avoid leaking of DNS requests
Otherwise:
http/cancel_by_host_ns_timeout_inactive_server: [msg] Nameserver 127.0.0.1:37035 has failed: request timed out.
[msg] All nameservers have failed
OK
1 tests ok. (0 skipped)
==26211==
==26211== FILE DESCRIPTORS: 3 open at exit.
==26211== Open file descriptor 2: /dev/pts/47
==26211== <inherited from parent>
==26211==
==26211== Open file descriptor 1: /dev/pts/47
==26211== <inherited from parent>
==26211==
==26211== Open file descriptor 0: /dev/pts/47
==26211== <inherited from parent>
==26211==
==26211==
==26211== HEAP SUMMARY:
==26211== in use at exit: 1,112 bytes in 5 blocks
==26211== total heap usage: 149 allocs, 144 frees, 18,826 bytes allocated
==26211==
==26211== 40 bytes in 1 blocks are indirectly lost in loss record 1 of 5
==26211== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211== by 0x498F5B: evbuffer_add_cb (buffer.c:3309)
==26211== by 0x4A0EF5: bufferevent_socket_new (bufferevent_sock.c:366)
==26211== by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211== by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211== by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211== by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211== by 0x491699: tinytest_main (tinytest.c:434)
==26211== by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 136 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==26211== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211== by 0x491FF0: evbuffer_new (buffer.c:365)
==26211== by 0x49A1BE: bufferevent_init_common_ (bufferevent.c:300)
==26211== by 0x4A0E44: bufferevent_socket_new (bufferevent_sock.c:353)
==26211== by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211== by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211== by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211== by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211== by 0x491699: tinytest_main (tinytest.c:434)
==26211== by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 136 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==26211== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211== by 0x491FF0: evbuffer_new (buffer.c:365)
==26211== by 0x49A1FB: bufferevent_init_common_ (bufferevent.c:305)
==26211== by 0x4A0E44: bufferevent_socket_new (bufferevent_sock.c:353)
==26211== by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211== by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211== by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211== by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211== by 0x491699: tinytest_main (tinytest.c:434)
==26211== by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 536 bytes in 1 blocks are indirectly lost in loss record 4 of 5
==26211== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211== by 0x4A0E15: bufferevent_socket_new (bufferevent_sock.c:350)
==26211== by 0x4BFADF: evhttp_connection_base_bufferevent_new (http.c:2375)
==26211== by 0x4BFC8F: evhttp_connection_base_new (http.c:2427)
==26211== by 0x460DAA: http_cancel_test (regress_http.c:1417)
==26211== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211== by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211== by 0x491699: tinytest_main (tinytest.c:434)
==26211== by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== 1,112 (264 direct, 848 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==26211== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==26211== by 0x4AAEB2: event_mm_calloc_ (event.c:3459)
==26211== by 0x4D0564: evdns_getaddrinfo (evdns.c:4685)
==26211== by 0x4B13BA: evutil_getaddrinfo_async_ (evutil.c:1575)
==26211== by 0x4A139E: bufferevent_socket_connect_hostname (bufferevent_sock.c:524)
==26211== by 0x4C02DB: evhttp_connection_connect_ (http.c:2588)
==26211== by 0x4C04DD: evhttp_make_request (http.c:2643)
==26211== by 0x4615FF: http_cancel_test (regress_http.c:1504)
==26211== by 0x490A78: testcase_run_bare_ (tinytest.c:105)
==26211== by 0x490D5A: testcase_run_one (tinytest.c:252)
==26211== by 0x491699: tinytest_main (tinytest.c:434)
==26211== by 0x47E0E0: main (regress_main.c:461)
==26211==
==26211== LEAK SUMMARY:
==26211== definitely lost: 264 bytes in 1 blocks
==26211== indirectly lost: 848 bytes in 4 blocks
==26211== possibly lost: 0 bytes in 0 blocks
==26211== still reachable: 0 bytes in 0 blocks
==26211== suppressed: 0 bytes in 0 blocks
Azat Khuzhin [Tue, 22 Mar 2016 20:36:19 +0000 (23:36 +0300)]
http: avoid leaking of fd in evhttp_connection_free()
Since we do close fd there if we don't have BEV_OPT_CLOSE_ON_FREE, and
evcon->fd can be incorrect (non -1), so just get it from the underlying
bufferevent to fix this.
And after this patch the following tests report 0 instead of 2307 fd leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
==11299== FILE DESCRIPTORS: 3 open at exit.
Azat Khuzhin [Tue, 22 Mar 2016 16:29:50 +0000 (19:29 +0300)]
http: get fd from be layer during connection reset
Since it can be non -1, and we must close it, otherwise we will have problems.
And after this patch the following tests report fd 2307 instead of 2309 fd leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
==10853== FILE DESCRIPTORS: 2307 open at exit.
Azat Khuzhin [Mon, 21 Mar 2016 14:09:17 +0000 (17:09 +0300)]
be_sock: cancel in-progress dns requests
Before this patch we didn't have such functionality and this can cause some
issues when we are cancelling HTTP request while after this we will get a
response/timeout from the NS and in this case error_cb will be called and can
damage newly started HTTP request.
We could also have problems with connect, but we don't have them since we
closes the fd in HTTP layer.
This is not so good that this is done in be_sock, but it is internal, so we can
change without pain. Plus I don't like that callback-via-evutil, but since we
have event_extra we need do like that.
And after this patch the following tests doesn't report leaks:
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
...
==10469== FILE DESCRIPTORS: 2309 open at exit.
...
==10469== HEAP SUMMARY:
==10469== in use at exit: 0 bytes in 0 blocks
==10469== total heap usage: 33,846 allocs, 33,846 frees, 4,617,651 bytes allocated
==10469==
==10469== All heap blocks were freed -- no leaks are possible
v2: do under lock
v3: reset dns request
v4: ignore EVUTIL_EAI_CANCEL in regular callback
This patches not 100% for http layer, but more for be_sock, but it was simpler
to add them here, plus it also shows some bugs with fd leaking in http layer.
Right now we have next picture (we can also play with timeouts/attempts for
evdns to make tests fail, IOW to track the failures even without valgrind):
$ valgrind --leak-check=full --show-reachable=yes --track-fds=yes --error-exitcode=1 regress --no-fork http/cancel..
http/cancel: OK
http/cancel_by_host: OK
http/cancel_by_host_no_ns: [msg] Nameserver 127.0.0.1:42489 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_inactive_server: OK
http/cancel_inactive_server: OK
http/cancel_by_host_no_ns_inactive_server: [msg] Nameserver 127.0.0.1:51370 has failed: request timed out.
[msg] All nameservers have failed
OK
http/cancel_by_host_server_timeout: OK
http/cancel_server_timeout: OK
http/cancel_by_host_no_ns_server_timeout: [msg] Nameserver 127.0.0.1:45054 has failed: request timed out.
[msg] All nameservers have failed
OK
9 tests ok. (0 skipped)
==3202==
==3202== FILE DESCRIPTORS: 2309 open at exit.
...
==8403== HEAP SUMMARY:
==8403== in use at exit: 1,104 bytes in 5 blocks
==8403== total heap usage: 10,916 allocs, 10,911 frees, 1,458,818 bytes allocated
==8403==
==8403== 40 bytes in 1 blocks are indirectly lost in loss record 1 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x498E48: evbuffer_add_cb (buffer.c:3309)
==8403== by 0x4A0DE2: bufferevent_socket_new (bufferevent_sock.c:366)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 2 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x491EDD: evbuffer_new (buffer.c:365)
==8403== by 0x49A0AB: bufferevent_init_common_ (bufferevent.c:300)
==8403== by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 136 bytes in 1 blocks are indirectly lost in loss record 3 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x491EDD: evbuffer_new (buffer.c:365)
==8403== by 0x49A0E8: bufferevent_init_common_ (bufferevent.c:305)
==8403== by 0x4A0D31: bufferevent_socket_new (bufferevent_sock.c:353)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 528 bytes in 1 blocks are indirectly lost in loss record 4 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x4A0D02: bufferevent_socket_new (bufferevent_sock.c:350)
==8403== by 0x4BF8BA: evhttp_connection_base_bufferevent_new (http.c:2369)
==8403== by 0x4BFA6A: evhttp_connection_base_new (http.c:2421)
==8403== by 0x460CFC: http_cancel_test (regress_http.c:1413)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== 1,104 (264 direct, 840 indirect) bytes in 1 blocks are definitely lost in loss record 5 of 5
==8403== at 0x4C2BBD5: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8403== by 0x4AAD2D: event_mm_calloc_ (event.c:3459)
==8403== by 0x4D0326: evdns_getaddrinfo (evdns.c:4682)
==8403== by 0x4B1213: evutil_getaddrinfo_async_ (evutil.c:1568)
==8403== by 0x4A1255: bufferevent_socket_connect_hostname (bufferevent_sock.c:517)
==8403== by 0x4C00B6: evhttp_connection_connect_ (http.c:2582)
==8403== by 0x4C02B8: evhttp_make_request (http.c:2637)
==8403== by 0x4614EC: http_cancel_test (regress_http.c:1496)
==8403== by 0x490965: testcase_run_bare_ (tinytest.c:105)
==8403== by 0x490C47: testcase_run_one (tinytest.c:252)
==8403== by 0x491586: tinytest_main (tinytest.c:434)
==8403== by 0x47DFCD: main (regress_main.c:461)
==8403==
==8403== LEAK SUMMARY:
==8403== definitely lost: 264 bytes in 1 blocks
==8403== indirectly lost: 840 bytes in 4 blocks
==8403== possibly lost: 0 bytes in 0 blocks
==8403== still reachable: 0 bytes in 0 blocks
==8403== suppressed: 0 bytes in 0 blocks
Azat Khuzhin [Sun, 13 Mar 2016 10:05:25 +0000 (13:05 +0300)]
test/http: exit from the loop in the errorcb to wait cancellation
This will make cancellation tests more graceful, that said that error_cb can
not be called sometimes if you will break the loop in cancel.
Plus drop that define for function generations, since function body changed,
and it is not generic anymore, plus that macro didn't used by anyone else.
Azat Khuzhin [Fri, 11 Mar 2016 16:58:05 +0000 (19:58 +0300)]
http: fix EVHTTP_CON_READ_ON_WRITE_ERROR when it doesn't supported by OS
For example win32 doesn't accept such things (maybe via overloaded IO, I'm not
sure), also I looked into curl and seems that the behaviour is the same (IOW
like with EVHTTP_CON_READ_ON_WRITE_ERROR on linux/win32).
Fixes: https://ci.appveyor.com/project/nmathewson/libevent/build/2.1.5.216#L499 (win32) Fixes: 680742e1665b85487f10c0ef3df021e3b8e98634 ("http: read server response
even after server closed the connection")
v2: v0 was just removing that flag, i.e. make it deprecated and set_flags() will return -1
This patch set fixes and covers http client and "Expect: 100-continue"
functionality (plus increase coverage under some related options, to avoid
further regressions).
* http-client-fix-expect-100-continue-v6:
http: fix leaking of response_code_line
http: fix "Expect: 100-continue" client side
test/http: separate coverage for EVHTTP_CON_READ_ON_WRITE_ERROR
test/http: cover "Expect: 100-continue" client-server interaction
test/http: *lingering tests shouldn't have "Expect: 100-continue"
Azat Khuzhin [Fri, 11 Mar 2016 17:40:52 +0000 (20:40 +0300)]
http: fix leaking of response_code_line
Since now evhttp_parse_response_line() can be called twice because after
"HTTP/1.1 100 Continue" we can have "HTTP/1.1 200"
==29162== 9 bytes in 1 blocks are definitely lost in loss record 1 of 1
==29162== at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==29162== by 0x5CBF0A9: strdup (in /lib/x86_64-linux-gnu/libc-2.21.so)
==29162== by 0x4AA3AC: event_mm_strdup_ (event.c:3493)
==29162== by 0x4BD843: evhttp_parse_response_line (http.c:1680)
==29162== by 0x4BE333: evhttp_parse_firstline_ (http.c:2013)
==29162== by 0x4BEA4F: evhttp_read_firstline (http.c:2243)
==29162== by 0x4BC5F8: evhttp_read_cb (http.c:1136)
==29162== by 0x4993F1: bufferevent_run_readcb_ (bufferevent.c:233)
==29162== by 0x49FBC0: bufferevent_trigger_nolock_ (bufferevent-internal.h:392)
==29162== by 0x49FF10: bufferevent_readcb (bufferevent_sock.c:208)
==29162== by 0x4A474A: event_persist_closure (event.c:1580)
==29162== by 0x4A49F5: event_process_active_single_queue (event.c:1639)
Azat Khuzhin [Fri, 11 Mar 2016 10:08:28 +0000 (13:08 +0300)]
http: fix "Expect: 100-continue" client side
Instead of sending data always at the beginning of the request wait until the
server will respond with "HTTP/1.1 100 Continue".
Before this patch server do send "HTTP/1.1 100 Continue" but client always send
post data even without waiting server response.
P.S. this patch also touches some not 100% related tab-align issues.
Azat Khuzhin [Fri, 11 Mar 2016 10:55:41 +0000 (13:55 +0300)]
be_sock: unfreeze buffers on fd changing
Only bufferevent_sock have evbuffer_freeze()/evbuffer_unfreeze() & ctrl ops, so
we don't need to fix other bufferevents (be_pair doesn't have ctrl op).
Found during draining buffers in http layer, and hence 501-not-implemented
error in regress http/.. (with some custom hacking).
Azat Khuzhin [Wed, 9 Mar 2016 21:33:04 +0000 (00:33 +0300)]
cmake: fix adding of compiler flags, and now it will
- add_compiler_flags() must accept array IOW just ARGN will be enoough
- add_compiler_flags() called with variable name instead of it's value
P.S. and fix some alignments issues
P.P.S. more cmake issues expected since now CFLAGS actually works
P.P.P.S. some issues with cmake cache is possible, so just reset it
Azat Khuzhin [Wed, 9 Mar 2016 21:25:09 +0000 (00:25 +0300)]
Replace -Wswitch-enum with -Wswitch, and add it into cmake rules too
According to https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html:
-Wswitch-enum
Warn whenever a switch statement has an index of enumerated type and lacks a
case for one or more of the named codes of that enumeration. case labels
outside the enumeration range also provoke warnings when this option is used.
The only difference between -Wswitch and this option is that this option
gives a warning about an omitted enumeration code even if there is a *default
label*.
Azat Khuzhin [Wed, 9 Mar 2016 15:55:20 +0000 (18:55 +0300)]
Merge branch 'more-graceful-http-v10'/lingering close
In short: now `evhttp_set_max_body_size()` is browser-friendly.
This patch set implements lingering close (something like nginx have), this
will make web-server more graceful for currently existing web browsers, so:
- without EVHTTP_CON_LINGERING_CLOSE/EVHTTP_SERVER_LINGERING_CLOSE
before: it will read min(max_body_size, Content-Length)
- with EVHTTP_CON_LINGERING_CLOSE/EVHTTP_SERVER_LINGERING_CLOSE:
it will read max(max_body_size, Content-Length), and web browsers will show
"413 Request Entity Too Large".
Also it fixes a bug on client-side with non-lingering close web-servers
(EVHTTP_CON_READ_ON_WRITE_ERROR), found during implementing web-server
lingering close.
* more-graceful-http-v10:
test: http/lingering_close: cover EVHTTP_SERVER_LINGERING_CLOSE
test: http/non_lingering_close: cover ~EVHTTP_SERVER_LINGERING_CLOSE
test: http/*: update expected HTTP codes for body exceeds `max_body_size`
http: take EVHTTP_CON_LINGERING_CLOSE into account for "Expect: 100-Continue"
test: http/data_length_constrains: set EVHTTP_CON_READ_ON_WRITE_ERROR
http: lingering close (like nginx have) for entity-too-large
http: read server response even after server closed the connection
test: increase buffer size for http/data_length_constraints to trigger EPIPE
Azat Khuzhin [Mon, 15 Feb 2016 00:26:40 +0000 (03:26 +0300)]
http: take EVHTTP_CON_LINGERING_CLOSE into account for "Expect: 100-Continue"
Also since after this patch code became more generic, we now respond with
HTTP_ENTITYTOOLARGE even without "Expect: 100-Continue", which is correct by
RFC.
Azat Khuzhin [Sun, 14 Feb 2016 21:12:54 +0000 (00:12 +0300)]
http: lingering close (like nginx have) for entity-too-large
By lingering close I mean something what nginx have for this name, by this term
I mean that we need to read all the body even if it's size greater then
`max_body_size`, otherwise browsers on win32 (including chrome) failed read the
http status - entity-too-large (while on linux chrome for instance are good),
and also this includes badly written http clients.
Refs: #321
v2: do this only under EVHTTP_SERVER_LINGERING_CLOSE
Azat Khuzhin [Wed, 10 Feb 2016 11:43:18 +0000 (14:43 +0300)]
http: read server response even after server closed the connection
Otherwise if we will try to write more data than server can accept
(see `evhttp_set_max_body_size()` for libevent server) we will get `EPIPE` and
will not try to read server's response which must contain 400 error for now
(which is not strictly correct though, it must 413).
```
$ strace regress --no-fork http/data_length_constraints
...
connect(10, {sa_family=AF_INET, sin_port=htons(43988), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
...
writev(10, [{"POST / HTTP/1.1\r\nHost: somehost\r"..., 60}, {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16324}], 2) = 16384
epoll_wait(5, [{EPOLLOUT, {u32=10, u64=10}}, {EPOLLIN, {u32=11, u64=11}}], 32, 50000) = 2
writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = 16384
ioctl(11, FIONREAD, [32768]) = 0
readv(11, [{"POST / HTTP/1.1\r\nHost: somehost\r"..., 4096}], 1) = 4096
epoll_ctl(5, EPOLL_CTL_DEL, 11, 0x7fff09d41e50) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 11, {EPOLLOUT, {u32=11, u64=11}}) = 0
epoll_wait(5, [{EPOLLOUT, {u32=10, u64=10}}, {EPOLLOUT, {u32=11, u64=11}}], 32, 50000) = 2
writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = 16384
writev(11, [{"HTTP/1.1 400 Bad Request\r\nConten"..., 129}, {"<HTML><HEAD>\n<TITLE>400 Bad Requ"..., 94}], 2) = 223
epoll_ctl(5, EPOLL_CTL_DEL, 11, 0x7fff09d42080) = 0
shutdown(11, SHUT_WR) = 0
close(11) = 0
epoll_wait(5, [{EPOLLOUT|EPOLLERR|EPOLLHUP, {u32=10, u64=10}}], 32, 50000) = 1
writev(10, [{"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., 16384}], 1) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=13954, si_uid=1000} ---
epoll_ctl(5, EPOLL_CTL_DEL, 10, 0x7fff09d42010) = 0
shutdown(10, SHUT_WR) = -1 ENOTCONN (Transport endpoint is not connected)
close(10) = 0
write(1, "\n FAIL ../test/regress_http.c:3"..., 37
```
Careful reader can ask why it send error even when it didn't read
`evcon->max_body_size`, and the answer will be checks for `evcon->max_body_size
against `Content-Length` header, which contains ~8MB (-2 bytes).
And also if we will not drain the output buffer than we will send buffer that
we didn't send in previous request and instead of sending method via
`evhttp_make_header()`.
Fixes: http/data_length_constraints
Refs: #321
v2: do this only under EVHTTP_CON_READ_ON_WRITE_ERROR flag
Azat Khuzhin [Sun, 14 Feb 2016 23:59:40 +0000 (02:59 +0300)]
http: fix conflicts EVHTTP_CON_AUTOFREE and EVHTTP_CON_REUSE_CONNECTED_ADDR
And we can't make them continuous, since the latest is a public API, and
otherwise we will break binary compatibility.
Also extra check for EVHTTP_CON_PUBLIC_FLAGS_END, in case somebody forgot about
this (implementer I mean).
Azat Khuzhin [Mon, 1 Feb 2016 14:32:09 +0000 (17:32 +0300)]
evdns: name_parse(): fix remote stack overread
@asn-the-goblin-slayer:
"the name_parse() function in libevent's DNS code is vulnerable to a buffer overread.
971 if (cp != name_out) {
972 if (cp + 1 >= end) return -1;
973 *cp++ = '.';
974 }
975 if (cp + label_len >= end) return -1;
976 memcpy(cp, packet + j, label_len);
977 cp += label_len;
978 j += label_len;
No check is made against length before the memcpy occurs.
This was found through the Tor bug bounty program and the discovery should be credited to 'Guido Vranken'."
Reproducer for gdb (https://gist.github.com/azat/e4fcf540e9b89ab86d02):
set $PROT_NONE=0x0
set $PROT_READ=0x1
set $PROT_WRITE=0x2
set $MAP_ANONYMOUS=0x20
set $MAP_SHARED=0x01
set $MAP_FIXED=0x10
set $MAP_32BIT=0x40
start
set $length=202
# overread
set $length=2
# allocate with mmap to have a seg fault on page boundary
set $l=(1<<20)*2
p mmap(0, $l, $PROT_READ|$PROT_WRITE, $MAP_ANONYMOUS|$MAP_SHARED|$MAP_32BIT, -1, 0)
set $packet=(char *)$1+$l-$length
# hack the packet
set $packet[0]=63
set $packet[1]='/'
p malloc(sizeof(int))
set $idx=(int *)$2
set $idx[0]=0
set $name_out_len=202
p malloc($name_out_len)
set $name_out=$3
# have WRITE only mapping to fail on read
set $end=$1+$l
p (void *)mmap($end, 1<<12, $PROT_NONE, $MAP_ANONYMOUS|$MAP_SHARED|$MAP_FIXED|$MAP_32BIT, -1, 0)
set $m=$4
p name_parse($packet, $length, $idx, $name_out, $name_out_len)
x/2s (char *)$name_out
Azat Khuzhin [Sat, 30 Jan 2016 21:57:16 +0000 (00:57 +0300)]
evutil_parse_sockaddr_port(): fix buffer overflow
@asn-the-goblin-slayer:
"Length between '[' and ']' is cast to signed 32 bit integer on line 1815. Is
the length is more than 2<<31 (INT_MAX), len will hold a negative value.
Consequently, it will pass the check at line 1816. Segfault happens at line
1819.
Generate a resolv.conf with generate-resolv.conf, then compile and run
poc.c. See entry-functions.txt for functions in tor that might be
vulnerable.
Please credit 'Guido Vranken' for this discovery through the Tor bug bounty
program."
Reproducer for gdb (https://gist.github.com/azat/be2b0d5e9417ba0dfe2c):
start
p (1ULL<<31)+1ULL
# $1 = 2147483649
p malloc(sizeof(struct sockaddr))
# $2 = (void *) 0x646010
p malloc(sizeof(int))
# $3 = (void *) 0x646030
p malloc($1)
# $4 = (void *) 0x7fff76a2a010
p memset($4, 1, $1)
# $5 = 1990369296
p (char *)$4
# $6 = 0x7fff76a2a010 '\001' <repeats 200 times>...
set $6[0]='['
set $6[$1]=']'
p evutil_parse_sockaddr_port($4, $2, $3)
# $7 = -1
Azat Khuzhin [Thu, 7 Jan 2016 14:51:40 +0000 (17:51 +0300)]
cmake: Fix detection of ssize_t/SSIZE_T
Since ssize_it is POSIX, windows/VS also have this but with BaseTsd.h, plus the
logic prefers "ssize_t" (lower) instead of "SSIZE_T" (upper) when the latest
only available -- fix this too.
Azat Khuzhin [Sat, 2 Jan 2016 21:23:22 +0000 (00:23 +0300)]
test/regress: cover event_del() waiting mechanism
Since we have some issues (see refs) for changing waiting order in event_del()
I wrote this simple test, so maybe this test can explain something or at least
cover what we have before and show it will be broken.
P.S. we really need avoid such stuff like lets-test-with-sleep/usleep.
Azat Khuzhin [Sun, 27 Dec 2015 13:49:42 +0000 (16:49 +0300)]
Merge branch 'event_reinit-for-signals-v3'
* event_reinit-for-signals-v3:
test/regress: cover existing signal callbacks and fork() + event_reinit()
test/regress: cover signals after fork() + event_reinit()
test/regress: main/fork: rewrite assertions by just removing event in callback
event_reinit: make signals works after fork() without evsig_add()
event_reinit: always re-init signal's socketpair
Azat Khuzhin [Sat, 26 Dec 2015 23:31:03 +0000 (02:31 +0300)]
test/regress: main/fork: rewrite assertions by just removing event in callback
Instead of assigning some variable value (got_child), and schedule exit from
loop from that callback, just remove event for that signal, and event loop will
exit automatically when there will be no events.
event_reinit: make signals works after fork() without evsig_add()
event_reinit() removes the event, but only evsig_add puts it back. So any
signals set up before event_reinit will be ignored until another signal is
added.
Before this patch event_reinit() only closes the signal socketpair fds and
recreates them if signals have been added, but this is wrong, since socketpair
fds created on backend init, and if we will not re-create them bad things in
child/parent signal handling will happens (and indeed this is what happens for
non-reinit backends like select).
billsegall [Thu, 17 Dec 2015 11:27:37 +0000 (21:27 +1000)]
Provide a mechanism for building the library on Windows with different compiler flags. Add a batch file that builds it for the M[DT][d] options and performs a hunt and gather of the different output libraries.
billsegall [Wed, 16 Dec 2015 01:17:36 +0000 (11:17 +1000)]
The Windows socket type is defined as SOCKET.
Under the hood it's an unsigned rather than a signed type and whilst C
compilers are largely happy with this C++ compilers tend to be fussy
about class function signatures which makes C++ usage of libevent
problematic.
Azat Khuzhin [Mon, 23 Nov 2015 10:52:31 +0000 (13:52 +0300)]
be_sock: bufferevent_socket_connect_hostname(): make it thread-safe
If you use bufferevent_socket_connect_hostname() to resolve, then ipv4 answer
can be returned before ipv6 scheduled and if you will destroy bufferevent after
ipv4 answer will come (in a separate thread of course) then ipv6 will trigger
UAF:
$ a.out
=================================================================
==29733==ERROR: AddressSanitizer: heap-use-after-free on address 0x60200000ef50 at pc 0x0000004b7aef bp 0x7fffffffd940 sp 0x7fffffffd0f8
READ of size 2 at 0x60200000ef50 thread T0
#0 0x4b7aee in __interceptor_index (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4b7aee)
#1 0x5060eb in string_num_dots /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2739
#2 0x5078df in search_request_new /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:3214
#3 0x506afd in evdns_base_resolve_ipv6 /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:2935
#4 0x50aa94 in evdns_getaddrinfo /src/oss/libevent/libevent-github/.cmake-debug/../evdns.c:4719
#5 0x51de4f in evutil_getaddrinfo_async_ /src/oss/libevent/libevent-github/.cmake-debug/../evutil.c:1567
#6 0x4fe023 in bufferevent_socket_connect_hostname /src/oss/libevent/libevent-github/.cmake-debug/../bufferevent_sock.c:519
#7 0x524f54 in evhttp_connection_connect_ /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2493
#8 0x525156 in evhttp_make_request /src/oss/libevent/libevent-github/.cmake-debug/../http.c:2548
#9 0x52d373 in main (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d373)
#10 0x7ffff6849b44 in __libc_start_main /tmp/buildd/glibc-2.19/csu/libc-start.c:287
#11 0x445806 in _start (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x445806)
0x60200000ef50 is located 0 bytes inside of 15-byte region [0x60200000ef50,0x60200000ef5f)
freed by thread T1 here:
#0 0x4cc4f2 in __interceptor_free (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x4cc4f2)
#1 0x5141c1 in event_mm_free_ /src/oss/libevent/libevent-github/.cmake-debug/../event.c:3512
#2 0x522402 in evhttp_connection_free /src/oss/libevent/libevent-github/.cmake-debug/../http.c:1206
#3 0x52cc5f in connection_closer (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52cc5f)
#4 0x50e80e in event_process_active_single_queue /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1642
#5 0x50ed57 in event_process_active /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1734
#6 0x50f458 in event_base_loop /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1957
#7 0x50eddf in event_base_dispatch /src/oss/libevent/libevent-github/.cmake-debug/../event.c:1768
#8 0x52d075 in event_dispatch_thread (/src/oss/libevent/libevent-github/.invest/217-evhttp-threaded/a.out+0x52d075)
#9 0x7ffff74fc0a3 in start_thread /tmp/buildd/glibc-2.19/nptl/pthread_create.c:309
Azat Khuzhin [Wed, 25 Nov 2015 14:47:42 +0000 (17:47 +0300)]
autotools: fix getservbyname() detection
The mentioned commit adds this check under-the-else of the previous condition
between getaddrinfo()/gethostbyname_r(), so this check is triggered only when
we don't have getaddrinfo() which is wrong, fix this by move it upper.
Fixes [CI] since it uses getservbyname() and it failed with autotools only
(cmake detection is ok).
CI: https://travis-ci.org/libevent/libevent/builds/93125954 Fixes: af08a94085e49e6942835b4c6b50a774536d5b5b ("Check for getservbyname even
if not on win32.")
Azat Khuzhin [Wed, 25 Nov 2015 10:09:15 +0000 (13:09 +0300)]
Merge branch 'evdns-fail-requests-v6'
This patchset fixes some issues wit evdns_base_free(..., fail_requests=1), and
there are two cases: with callback wrapper for evdns_getaddrinfo() and not,
both fixed by this patches, and adds regression tests for this.
* evdns-fail-requests-v6:
evdns: evdns_base_free(): fix UAF of evdns_base with @fail_requests
test/dns: cover evdns_getaddrinfo() and evdns_base_free() with @fail_requests
evdns: evdns_base_free(): free requests before namservers
test/dns: cover @fail_requests for evdns_base_free()
test/dns: more graceful coverage of @fail_requests
Azat Khuzhin [Mon, 23 Nov 2015 12:05:19 +0000 (15:05 +0300)]
evdns: evdns_base_free(): fix UAF of evdns_base with @fail_requests
If you call evdns_base_free() with @fail_requests == 1, then it will defer
callback with DNS_ERR_SHUTDOWN, but that callback (internal) uses
data->evdns_base, but we already freed that evdns base, so we can't do
this, fix this by checking @result to DNS_ERR_SHUTDOWN.
Azat Khuzhin [Mon, 23 Nov 2015 12:36:30 +0000 (15:36 +0300)]
evdns: evdns_base_free(): free requests before namservers
Otherwise we will trigger next UAF:
$ valgrind --vgdb-error=1 regress --no-fork +dns/client_fail_requests
==24733== Memcheck, a memory error detector
==24733== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24733== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==24733== Command: regress --no-fork +dns/client_fail_requests
==24733==
==24733==
==24733== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==24733== /path/to/gdb regress
==24733== and then give GDB the following command
==24733== target remote | /usr/lib/valgrind/../../bin/vgdb --pid=24733
==24733== --pid is optional if only one valgrind process is running
==24733==
dns/client_fail_requests: ==24733== Invalid read of size 4
==24733== at 0x4C3352: request_finished (evdns.c:662)
==24733== by 0x4CC8B7: evdns_base_free_and_unlock (evdns.c:4048)
==24733== by 0x4CCAFD: evdns_base_free (evdns.c:4088)
==24733== by 0x458E95: dns_client_fail_requests_test (regress_dns.c:2039)
==24733== by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733== by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733== by 0x48F67E: tinytest_main (tinytest.c:434)
==24733== by 0x47C0DA: main (regress_main.c:461)
==24733== Address 0x61e6f70 is 448 bytes inside a block of size 456 free'd
==24733== at 0x4C29EAB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24733== by 0x4A8F4D: event_mm_free_ (event.c:3512)
==24733== by 0x4CC7A1: evdns_nameserver_free (evdns.c:4021)
==24733== by 0x4CC7DC: evdns_base_free_and_unlock (evdns.c:4037)
==24733== by 0x4CCAFD: evdns_base_free (evdns.c:4088)
==24733== by 0x458E95: dns_client_fail_requests_test (regress_dns.c:2039)
==24733== by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733== by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733== by 0x48F67E: tinytest_main (tinytest.c:434)
==24733== by 0x47C0DA: main (regress_main.c:461)
==24733== Block was alloc'd at
==24733== at 0x4C28C4F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24733== by 0x4A8D5A: event_mm_malloc_ (event.c:3437)
==24733== by 0x4C8B96: evdns_nameserver_add_impl_ (evdns.c:2505)
==24733== by 0x4C916D: evdns_base_nameserver_ip_add (evdns.c:2629)
==24733== by 0x458DA3: dns_client_fail_requests_test (regress_dns.c:2031)
==24733== by 0x48EA5D: testcase_run_bare_ (tinytest.c:105)
==24733== by 0x48ED3F: testcase_run_one (tinytest.c:252)
==24733== by 0x48F67E: tinytest_main (tinytest.c:434)
==24733== by 0x47C0DA: main (regress_main.c:461)
==24733==
==24733== (action on error) vgdb me ...
Azat Khuzhin [Mon, 23 Nov 2015 12:14:32 +0000 (15:14 +0300)]
test/dns: more graceful coverage of @fail_requests
In case when evdns_base_free() called with @fail_requests, we can potentially
have leaks, but we can avoid them if we will run event loop once again to
trigger defer cbs, so let's do this, instead of magical decrements (and also
this will give an example how to avoid leaks for evdns).
projects / libevent / log