3
0

tcp.txt 3.2 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293
  1. Some less-widely known details of TCP connections.
  2. Properly closing the connection.
  3. After this code sequence:
  4. sock = socket(AF_INET, SOCK_STREAM, 0);
  5. connect(sock, &remote, sizeof(remote));
  6. write(sock, buffer, 1000000);
  7. a large block of data is only buffered by kernel, it can't be sent all at once.
  8. What will happen if we close the socket?
  9. "A host MAY implement a 'half-duplex' TCP close sequence, so that
  10. an application that has called close() cannot continue to read
  11. data from the connection. If such a host issues a close() call
  12. while received data is still pending in TCP, or if new data is
  13. received after close() is called, its TCP SHOULD send a RST
  14. to show that data was lost."
  15. IOW: if we just close(sock) now, kernel can reset the TCP connection
  16. (send RST packet).
  17. This is problematic for two reasons: it discards some not-yet sent
  18. data, and it may be reported as error, not EOF, on peer's side.
  19. What can be done about it?
  20. Solution #1: block until sending is done:
  21. /* When enabled, a close(2) or shutdown(2) will not return until
  22. * all queued messages for the socket have been successfully sent
  23. * or the linger timeout has been reached.
  24. */
  25. struct linger {
  26. int l_onoff; /* linger active */
  27. int l_linger; /* how many seconds to linger for */
  28. } linger;
  29. linger.l_onoff = 1;
  30. linger.l_linger = SOME_NUM;
  31. setsockopt(sock, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger));
  32. close(sock);
  33. Solution #2: tell kernel that you are done sending.
  34. This makes kernel send FIN after all data is written:
  35. shutdown(sock, SHUT_WR);
  36. close(sock);
  37. However, experiments on Linux 3.9.4 show that kernel can return from
  38. shutdown() and from close() before all data is sent,
  39. and if peer sends any data to us after this, kernel still responds with
  40. RST before all our data is sent.
  41. In practice the protocol in use often does not allow peer to send
  42. such data to us, in which case this solution is acceptable.
  43. Solution #3: if you know that peer is going to close its end after it sees
  44. our FIN (as EOF), it might be a good idea to perform a read after shutdown().
  45. When read finishes with 0-sized result, we conclude that peer received all
  46. the data, saw EOF, and closed its end.
  47. However, this incurs small performance penalty (we run for a longer time)
  48. and requires safeguards (nonblocking reads, timeouts etc) against
  49. malicious peers which don't close the connection.
  50. Solutions #1 and #2 can be combined:
  51. /* ...set up struct linger... then: */
  52. setsockopt(sock, SOL_SOCKET, SO_LINGER, &linger, sizeof(linger));
  53. shutdown(sock, SHUT_WR);
  54. /* At this point, kernel sent FIN packet, not RST, to the peer, */
  55. /* even if there is buffered read data from the peer. */
  56. close(sock);
  57. Defeating Nagle.
  58. Method #1: manually control whether partial sends are allowed:
  59. This prevents partially filled packets being sent:
  60. int state = 1;
  61. setsockopt(fd, IPPROTO_TCP, TCP_CORK, &state, sizeof(state));
  62. and this forces last, partially filled packet (if any) to be sent:
  63. int state = 0;
  64. setsockopt(fd, IPPROTO_TCP, TCP_CORK, &state, sizeof(state));
  65. Method #2: make any write to immediately send data, even if it's partial:
  66. int state = 1;
  67. setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &state, sizeof(state));