Description

Interview with Thomas Graf of Cisco, regarding the Cilium project.

Cilium is a “science project” that Thomas and others at Cisco and
elsewhere are hacking on, to address the question of how to address
policy in a legacy-free container environment that scales to millions of
endpoints. It's an experiment because the outcome isn't yet certain, and
it's a question that hasn't seen much work outside of hyperscale
providers.

Cilium is based on eBPF, a
Linux kernel technology that introduces the ability for userspace to
inject custom programs into the kernel using a bytecode analogous to Java
virtual machine bytecode. Cilium uses eBPF-based hooks can intercept
packets at various places in their path through the kernel to implement a
flexible policy engine.

Topics include:

• 
  How Chris Wright encouraged Thomas to become involved with Open
  vSwitch, when Thomas was at Red Hat.
• 
  The important differences between containers and VMs (quantity,
  duration of workloads, and frequency of state changes) and how Cilium
  addresses these issues.
• 
  The toolchain that Cilium uses to generate a eBPF program customized
  for the policy of each container in a minimal and complete way.
• 
  The potential to bypass kernel sk_buff overhead by
  intercepting packets before these data structures are constructed, via
  Express Data Path (XDP), leading to the potential for DPDK-like packet
  forwarding performance without ever leaving the kernel. What's the
  drawback? We don't know yet how it will play out.
• 
  The main benefit of getting XDP-based early access to packets is to
  avoid the main Linux networking code paths, which are optimized for
  delivery to socket buffers (taking advantage of segmentation offload)
  as opposed to forwarding.
• 
  Dropping packets quickly is important because many operators are under
  attack all the time.
• 
  Importance of performance for small and large packets.
• 
  Languages available for writing eBPF: C via LLVM/Clang and Python
  (among others?).
• 
  Limitations due to the verifier (primarily restrictions on loops), and
  how to work around them.
• 
  How Cilium generates its eBPF programs: a base C program, plus an agent
  in Go that generates a C header file. Cilium compiles the C program to
  eBPF bytecode, using LLVM, then loads it into the running kernel with
  the tc utility.
• 
  Potential for difficulty in getting a compiler toolchain into
  production deployments. Is the simplicity of supplying Cilium as a
  container image that builds in the toolchain an advantage?
• 
  How often eBPF programs need to be recompiled in Cilium.
• 
  eBPF “map” data structures that can be shared among eBPF programs,
  the rest of the kernel, and userspace.
• 
  Policy in Cilium. The basic idea is that whoever specifies Cilium
  policy should not have to understand traditional networking concepts
  like IP addresses ands port. Instead, abstract labels specify which
  classes of containers can talk to each other.
• 
  Lessons learned from policy in Cilium.
• 
  How Cilium does datapath packet processing, how it passes labels from
  source to destination, and where it applies policy.
• 
  The direction in which Cilium points for eBPF support in Open vSwitch:
  first, it shows that it is possible; second, it shows that the tracing
  buffer mechanism available from eBPF is a potential replacement for
  Open vSwitch “upcalls” currently implemented via Netlink; third, it
  points out an alternative for the flow-based model. (Does it makes
  sense to implement OVN directly via code generation?)
• 
  Connection tracking in eBPF.
• 
  eBPF helper functions in Linux, and the limitations of the current
  ones.
• 
  Potential for applying eBPF to other targets such as DPDK or the OVS
  port to Hyper-V.
• 
  Performance penalty for eBPF versus native code.
• 
  Early controversy in the kernel community over eBPF when it was
  introduced.
• 
  What's next for Cilium: load balancing, IPsec, IPv4 .

More information about Cilium: slides
and the code repository.

You can find Thomas on the ovs-dev mailing
list, @tgraf__ on Twitter,
or on Facebook.

OVS Orbit is produced by Ben Pfaff. The
intro and bumper music is Electro
Deluxe, featuring Gurdonack, copyright 2014 by My Free Mickey. The
outro music is Girls like
you, featuring Thespinwires, copyright 2014 by Stefan Kartenberg.
All content is licensed under a Creative Commons Attribution 3.0
Unported (CC BY 3.0) license.