Multiple Execution Contexts in lwIP code

The most common source of lwIP problems is to have multiple execution contexts inside the lwIP code.

lwIP can be used in two basic modes: Mainloop mode ("NO_SYS") (no OS/RTOS running on target system) or OS mode (TCPIP thread) (there is an OS running on the target system).

Mainloop Mode

In mainloop mode, only Callback-style APIs can be used. The user has two possibilities to ensure there is only one exection context at a time in lwIP:

1) Deliver RX ethernet packets directly in interrupt context to lwIP by calling netif->input directly in interrupt. This implies all lwIP callback functions are called in IRQ context, which may cause further problems in application code: IRQ is blocked for a long time, multiple execution contexts in application code etc. When the application wants to call lwIP, it only needs to disable interrupts during the call. If timers are involved, even more locking code is needed to lock out timer IRQ and ethernet IRQ from each other, assuming these may be nested.

2) Run lwIP in a mainloop. There is example code here: Mainloop mode ("NO_SYS"). lwIP is ONLY called from mainloop callstacks here. The ethernet IRQ has to put received telegrams into a queue which is polled in the mainloop. Ensure lwIP is NEVER called from an interrupt, e.g. some SPI IRQ wants to forward data to udp_send() or tcp_write()!

OS Mode

In OS mode, Callback-style APIs AND Sequential-style APIs can be used. Sequential-style APIs are designed to be called from threads other than the TCPIP thread, so there is nothing to consider here. But Callback-style APIs functions must ONLY be called from TCPIP thread. It is a common error to call these from other threads or from IRQ contexts. Ethernet RX needs to deliver incoming packets in the correct way by sending a message to TCPIP thread, this is implemented in tcpip_input(). Again, ensure lwIP is NEVER called from an interrupt, e.g. some SPI IRQ wants to forward data to udp_send() or tcp_write()!

1) tcpip_callback() can be used get called back from TCPIP thread, it is safe to call any Callback-style APIs from there.

2) Use LWIP_TCPIP_CORE_LOCKING. All Callback-style APIs functions can be called when lwIP core lock is aquired, see LOCK_TCPIP_CORE() and UNLOCK_TCPIP_CORE(). These macros cannot be used in an interrupt context! Note the OS must correctly handle priority inversion for this.