1
0

061-softirq-let-ksoftirqd-do-its-job.patch 2.6 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283
  1. From: Eric Dumazet <edumazet@google.com>
  2. Date: Wed, 31 Aug 2016 10:42:29 -0700
  3. Subject: [PATCH] softirq: let ksoftirqd do its job
  4. A while back, Paolo and Hannes sent an RFC patch adding threaded-able
  5. napi poll loop support : (https://patchwork.ozlabs.org/patch/620657/)
  6. The problem seems to be that softirqs are very aggressive and are often
  7. handled by the current process, even if we are under stress and that
  8. ksoftirqd was scheduled, so that innocent threads would have more chance
  9. to make progress.
  10. This patch makes sure that if ksoftirq is running, we let it
  11. perform the softirq work.
  12. Jonathan Corbet summarized the issue in https://lwn.net/Articles/687617/
  13. Tested:
  14. - NIC receiving traffic handled by CPU 0
  15. - UDP receiver running on CPU 0, using a single UDP socket.
  16. - Incoming flood of UDP packets targeting the UDP socket.
  17. Before the patch, the UDP receiver could almost never get cpu cycles and
  18. could only receive ~2,000 packets per second.
  19. After the patch, cpu cycles are split 50/50 between user application and
  20. ksoftirqd/0, and we can effectively read ~900,000 packets per second,
  21. a huge improvement in DOS situation. (Note that more packets are now
  22. dropped by the NIC itself, since the BH handlers get less cpu cycles to
  23. drain RX ring buffer)
  24. Since the load runs in well identified threads context, an admin can
  25. more easily tune process scheduling parameters if needed.
  26. Reported-by: Paolo Abeni <pabeni@redhat.com>
  27. Reported-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
  28. Signed-off-by: Eric Dumazet <edumazet@google.com>
  29. Cc: David Miller <davem@davemloft.net
  30. Cc: Jesper Dangaard Brouer <jbrouer@redhat.com>
  31. Cc: Peter Zijlstra <peterz@infradead.org>
  32. Cc: Rik van Riel <riel@redhat.com>
  33. ---
  34. --- a/kernel/softirq.c
  35. +++ b/kernel/softirq.c
  36. @@ -78,6 +78,17 @@ static void wakeup_softirqd(void)
  37. }
  38. /*
  39. + * If ksoftirqd is scheduled, we do not want to process pending softirqs
  40. + * right now. Let ksoftirqd handle this at its own rate, to get fairness.
  41. + */
  42. +static bool ksoftirqd_running(void)
  43. +{
  44. + struct task_struct *tsk = __this_cpu_read(ksoftirqd);
  45. +
  46. + return tsk && (tsk->state == TASK_RUNNING);
  47. +}
  48. +
  49. +/*
  50. * preempt_count and SOFTIRQ_OFFSET usage:
  51. * - preempt_count is changed by SOFTIRQ_OFFSET on entering or leaving
  52. * softirq processing.
  53. @@ -313,7 +324,7 @@ asmlinkage __visible void do_softirq(voi
  54. pending = local_softirq_pending();
  55. - if (pending)
  56. + if (pending && !ksoftirqd_running())
  57. do_softirq_own_stack();
  58. local_irq_restore(flags);
  59. @@ -340,6 +351,9 @@ void irq_enter(void)
  60. static inline void invoke_softirq(void)
  61. {
  62. + if (ksoftirqd_running())
  63. + return;
  64. +
  65. if (!force_irqthreads) {
  66. #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK
  67. /*