Reported on debian bugtracker here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724580
We can't remove an fdinfo from client_fdlist and still expect to access
the fdinfo via a pointer we got from get_fdinfo(&client_fdlist) since
rm_fd() modifies the data at the address pointed to. So instead of
removing it from the list and then adding it right back, we just don't
remove it in the first place.