The PBS/Torque scheduler that ships w/ Ubuntu 12.04 uses an interesting method to verify that user requests from a submission node cannot impersonate anyone else. In a nutshell, any Torque command (qsub, qstat, etc) calls a suid program (pbs_iff) which connects to the pbs server from a privileged port and notifies the server the client port and what user will be sending commands from that port. pbs_iff receives this information by looking at the source port on the file handle passed to if during its clone. The whole handshake looks like this:
- Unprivileged client opens a socket to the pbs server
- Client calls clone and passes the file handle number to a suid pbs_iff as an argument
- pbs_iff reads the source port off of the file handle
- pbs_iff opens a socket from a priviliged port to the pbs server and sends invoking user and source port .
- The pbs server now trusts that commands from the initial socket belong to the user passed by pbs_iff
- pbs_iff terminates and the original client sends whatever commands it desires.
This works nice in C where the default is to pass all file handles to the child process on a fork. However, many languages frown on this file handle leaking for a number of reasons and have decided this default is a bad idea. Java is one of these, so it nicely sets FD_CLOEXEC on all file handles it opens. This means when you use the ProcessBuilder or call Runtime.exec, you can’t see any file handles you previously had open thereby breaking Torque’s security mechanism.