Kambiz.N.A Personal Weblog: How to use Java to connect with HTTP servers outside your corporate firewall

Summary
This tip will show you how to write Java applications that can get past your corporate proxy and access Web servers on the Internet. Adding proxy support to your Java applications involves writing just a few additional lines of code and doesn't rely on any security "loopholes."

Almost every company is concerned with protecting its internal network from hackers and thieves. One common security measure is to completely disconnect the corporate network from the Internet. If the bad guys can't connect to any of your machines, they can't hack into them. The unfortunate side effect of this tactic is that internal users can't access external Internet servers, like Yahoo or JavaWorld. To address this problem, network administrators often install something called a "proxy server." Essentially, a proxy is a service that sits between the Internet and the internal network and manages connections between the two worlds. Proxies help reduce outside security threats while still allowing internal users to access Internet services. While Java makes it easy to write Internet clients, these clients are useless unless they can get past your proxy. Fortunately, Java makes it easy to work with proxies -- if you know the magic words, that is.

The secret to combining Java and proxies lies in activating certain system properties in the Java runtime. These properties appear to be undocumented, and are whispered between programmers as part of the Java folklore. In order to work with a proxy, your Java application needs to specify information about the proxy itself as well as specify user information for authentication purposes. In your program, before you begin to work with any Internet protocols, you'll need to add the following lines:

System.getProperties().put( "proxySet", "true" ); System.getProperties().put( "proxyHost", "myProxyMachineName" ); System.getProperties().put( "proxyPort", "85" );

The first line above tells Java that you'll be using a proxy for your connections, the second line specifies the machine that the proxy lives on, and the third line indicates what port the proxy is listening on. Some proxies require a user to type in a username and password before Internet access is granted. You've probably encountered this behavior if you use a Web browser behind a firewall. Here's how to perform the authentication:

URLConnection connection = url.openConnection(); String password = "username:password"; String encodedPassword = base64Encode( password ); connection.setRequestProperty( "Proxy-Authorization", encodedPassword );

The idea behind the above code fragment is that you must adjust your HTTP header to send out your user information. This is achieved with the setRequestProperty() call. This method allows you to manipulate the HTTP headers before the request is sent out. HTTP requires the user name and password to be base64 encoded. Luckily, there are a couple of public domain APIs that will perform the encoding for you (see the Resources section).

As you can see, there's not a whole lot to adding proxy support to your Java application. Given what you now know, and a little research (you'll have to find out how your proxy handles the protocol you're interested in and how to deal with user authentication), you can implement your proxy with other protocols.

Proxying FTP
Scott D. Taylor sent in the magic incantation to deal with proxying the FTP protocol:

defaultProperties.put( "ftpProxySet", "true" ); defaultProperties.put( "ftpProxyHost", "proxy-host-name" ); defaultProperties.put( "ftpProxyPort", "85" );

You can then access the files URLs using the "ftp" protocol via something like:

URL url = new URL("ftp://ftp.netscape.com/pub/navigator/3.04/windows/readme.txt" );

If anybody has examples of using a proxy with other Internet protocols, I'd love to see them.

Note: Has only been tested with JDK 1.1.4.

Follow-up Tips!

from Marla Bonar:

For those still using JDK 1.1.7 (with WebSphere 3.0), setting the system properties for proxyHost and proxyPort does not work; conn.getInputStream() either returns with Connection timed out or No route to host. However, I was able to work around this problem by using the URL constructor that takes the Host and Port as parameters (using my proxy host and port):

public URL(String protocol, String host, int port, String file).

from Dylan Walsh:

The method for providing authentication via username and password does not work. "Basic " should be at the start of the authentication string; for example:

String encodedPassword = base64Encode( password );

should be:

String encodedPassword = "Basic " + base64Encode( password );

Also you don't need a separate program to do the base64 encode. You can use the sun.misc.BASE64Encoder() class instead. Here is what the code looks like with both changes:

System.getProperties().put("proxySet", "true"); System.getProperties().put("proxyHost", proxyHost); System.getProperties().put("proxyPort", proxyPort); String authString = "userid:password"; String auth = "Basic " + new sun.misc.BASE64Encoder ().encode(authString.getBytes()); URL url = new URL("http://java.sun.com/"); URLConnection conn = url.openConnection(); conn.setRequestProperty("Proxy-Authorization", auth);

from Marcel Oerlemans:

Here's how you use Socks 4 proxy servers:

System.getProperty("socksProxySet", true); System.getProperty("socksProxyHost", proxyHostName); System.getProperty("socksProxyPort", proxyPort); Usually the proxyPort for Socks 4 is port 1080

You can then make your connection using Socks4.

Resources