FTP passive mode and Java Sockets

Okay, my first blog entry. Be gentle...

I just spent most of the day diagnosing this problem with an FTP server and a Java application, and I thought maybe I could save someone the effort some time.

So, first, the background.

Recall, FTP uses two TCP connections, a control connection (for commands) and a data connection (for data transfer). By default most FTP servers use Active mode for the data connection, which essentially turns the client into a server (client opens up a port and server makes a socket connection to it to do the data transfers). In Passive (PASV) mode however, the server opens up a port and the client connects to the listening server, but passive mode is usually optional (has to be set once the initial FTP connection is established). There's a good explanation of all of this (better than mine, and with pictures) here: http://www.slacksite.com/other/ftp.html

Now, how does this relate to Java Socket in particular?

Well, we were having a problem in our app where during a bunch of FTP activity, the app would just hang. We suspected thread deadlocking, but after we did a bunch of thread dumps while the app was in the hung state, there were no deadlocks to be seen. BTW, I found a cool Java thread dump translation/visualization tool here: http://ftpna2.bea.com/pub/downloads/ThreadDumpAnalyser.zip (parses thread dump files into XML, and then XSLs them into fancy html and graphviz graphs and stuff - neat!).

So, no thread deadlocks, but I wrote a unit test to isolate the problem (spins a bunch of threads up that do a bunch of FTP activity on the same directory and file). I run the test and it hangs in the same way the app does. Cool. So after looking at several thread dumps of the hanging test, I noticed that the ftp getter threads in the test were always hung on a call to Socket.accept(). This seemed weird to me at first (the ftp getters are clients, not servers), until I remembered about the active ftp stuff. Turns out my clients were in active ftp mode, trying to open a socket for the data connection, and something on the server was going wrong (it was sending 425 responses). Well, the default timeout for Java Socket.accept() is 0, which means forever. Which means my threads will hang forever, long after the server has forgotten about my connections. Woohoo.

So, I switched my ftp client code to use FTP Passive mode, and voila, everything was right with the world (well, the FTP server still spews 425s sometimes, but my threads recover gracefully now, without hanging forever). BTW, you can set the socket timeout to something other than 0, but in my case I didn't need to, because in passive mode my ftp clients don't need to open sockets for listening anymore (nor could I really, since I'm using Commons VFS and FTPClient under the hood, so I don't have direct access to the Sockets they create). Anyway, hope this saves you the trouble sometime...