?

Log in

No account? Create an account
Today's annoyance is the Linux IP stack - 'Twas brillig, and the slithy toves did gyre and gimble in the wabe [entries|archive|friends|userinfo]
Thomas

[ website | Beware the Jabberwock... ]
[ deviantArt | the-boggyb ]
[ FanFiction | Torkell ]
[ Tumblr | torkellr ]

Links
[Random links| BBC news | Vulture Central | Slashdot | Dangerous Prototypes | LWN | Raspberry Pi]
[Fellow blogs| a Half Empty Glass | the Broken Cube | The Music Jungle | Please remove your feet | A letter from home]
[Other haunts| Un4seen Developments | Jazz 2 Online | EmuTalk.net | Feng's shui]

Today's annoyance is the Linux IP stack [Friday 10th August 2012 at 8:41 pm]
Thomas

boggyb
[Tags|]
[Where |Watching the sunset from the train to London]
[Playing |Digitally Imported: Vocal Trance]

Today's annoyance is the Linux IP stack.

Take the following network layout:


This shows two servers: server A has only one network interface, while server B has two. Now let's assume that server B is running a useful service on 192.168.2.11 which server A wants to access. Since server A is only connected to the 192.168.1.x network, to get at this service it has to use the router.

One would expect server B to work out that because the service is running on a different subnet to server A, that it would also have to use the router for any replies. With the strong host model (where an IP address belongs to a specific interface) this would work perfectly and everyone would be happy.

The problem here is that Linux uses the weak host model, where a server has a set of IP addresses and it can use any address on any interface. This causes all sorts of fun problems even when you're expecting it. In this case, it results in server B sending the replies via the 192.168.1.11 interface but with a source address of 192.168.2.11. Now, you may be thinking that this isn't really a problem because the packets are still getting to the right place, but you would be wrong. The router only sees half the traffic, and if it's running some sort of stateful firewall then it will get very unhappy and break your network connection.

I'd run into that exact problem some time ago with a test network at work and managed to work around it by adding several explicit routing instructions to the network config (which I'll dig out and post at some point). However, that's not all. Today a colleague found another way in which Linux breaks things.

Linux has an option called "rp_filter". Turning this on causes Linux to check the source address of each packet it receives against the list of addresses it knows it can route to via that interface. And, of course, it breaks the network setup I've shown above.

The problem this time is that the routing table on server B says that to get to 192.168.1.x it must use the top network, to get to 192.168.2.x it must use the bottom network, and for anything else it uses the top network. So when it gets a packet from server A to 192.168.2.11 it checks the source address of 192.168.1.10 against the list of routes for the bottom interface, fails to find a matching address, and discards the packet. Notably, it discards the packet without sending any kind of error back to the sender, making this a right pain to debug.

For added fun, it turns out that the distro we use changed the default for this parameter sometime in the past year or so. So all our test systems that were built ages ago work fine, while the shiny new system someone just built on site ends up with weird networking problems because of this!
Link | Previous Entry | Share | Next Entry[ Penny for your thoughts? ]