For many decades, application software programmers have been using the venerable BSD sockets API to make their applications communicate over (at least IP based) networks. Linux has supported TCP/UDP sockets ever since it had a network stack back in the 1990s. While those socket system call APIs are simple and straight-forward, they were designed at a time when internet access happened over dial-up modems and LANs had no more than 10 MBit/s, if at all.
With today's Multi-Gigabit speeds even in consumer equipment and 40GE/100GE network interface cards for servers being a reality, using those 1980s BSD/POSIX socket interfaces comes with a huge performance penalty.
Some specific use cases like single-flow high-throughput TCP on an end-node have seen optimizations that are transparent to the user (TCP segmentation offloading). But there's only so far you can go with that.
Parts of the industry have proposed user-space network stacks built on DPDK - but then basically you do no longer use the Linux kernel network stack at all, and subsequently have none of its features. Yes, that can be fast, but Linux becomes nothing but a bootloader, and you have to implement everything from Ethernet to ARP and IP+TCP in your application.
The answer of the Linux kernel community over the last 5+ years has been various new mechanisms and interfaces in the Linux kernel that revolutionize the way how applications can achieve higher network I/O throughput - whether an end host (server/client) or a packet-forwarding router/bridge/firewall.
This talk provides a brief but deeply technical introduction into the problem space, the new mechanisms and their use cases.
While the talk discusses features of the Linux kernel, we do not discuss their internals; the focus is on how those mechanisms can be used by applications.