Howto: VFSLib - Apache Commons VFS Reloaded

The famous Apache Commons VFS library has been on the market for years now (current version is 2.0). It's established, widely-spread, and has saved a lot of Java developers a lot of programming effort when it comes to network functionality. Traditional network protocols like FTP (File Transfer Protocol) and SFTP (Secure FTP) can be deployed using a consistent and configurable framework. No matter what the used protocol is, files and directories on server machines can be accessed always with the same VFS classes and methods.

Let's take a look at a simple Java code snippet showing how to copy a FTP file over to a server that is connected with SFTP:

 
 // We use the default file system manager here
 DefaultFileSystemManager fsmanager = (DefaultFileSystemManager) VFS.getManager();
 
 // Connect to anonymous FTP server, copy file over to SFTP server
 String ftpuri = "ftp://kernel.org/pub/site/README";
 String sftpuri = "sftp://scott:tiger@mycompany.com:22/mydir/README";
 
 // SFTP needs a little configuring
 FileSystemOptions options = new FileSystemOptions();
 SftpFileSystemConfigBuilder builder = SftpFileSystemConfigBuilder.getInstance();
 builder.setStrictHostKeyChecking(options, "no"); // Loose checking
 builder.setUserDirIsRoot(options, false); // "/" is root
 
 try {
 // Open input stream for FTP file (read from)
 FileObject ftpobj = fsmanager.resolveFile(ftpuri); // Throws
 InputStream istream = ftpobj.getContent().getInputStream();
 
 // Open output stream for SFTP file (write into)
 FileObject sftpobj = fsmanager.resolveFile(sftpuri, options); // Throws
 OutputStream ostream = sftpobj.getContent().getOutputStream();
 
 // Copy bytes
 int len;
 byte[] buffer = new byte[1024];
 while ((len = istream.read(buffer)) != -1) {
 ostream.write(buffer, 0, len);
 }
 ostream.flush();
 ostream.close();
 istream.close();
 
 long size = sftpobj.getContent().getSize();
 boolean success = sftpobj.exists() && size == ftpobj.getContent().getSize();
 }
 catch (Exception e) { e.printStackTrace(); }
 

As you can see, it is really simple to access network files with Commons VFS. Files and directories are referenced with standard URL addresses that are flexible enough to cover the supported protocols. Some schemes like FTP require additional options like passive mode which can be easily specified using config builder classes. New providers e.g. for cloud file systems like Dropbox©, Google Drive©, or Amazon S3© can be added to the file system managers.

The advantages of the Apache Commons VFS 2.0 library at a glance

• Consistent and configurable framework to access local and network files
• Access resources from different schemes with the same classes like FileObject
• List folders from the classpath using the jar protocol (not possible with URL)
• Flexibility: New providers can be easily added e.g. for Dropbox (see VFSLib section)
• Platform independent: Compatible with Windows, Mac OS, Linux, ...

Adding Support For Recent File Systems With VFSLib

VFSLib is a Java library to extend the Apache Commons VFS 2.0 library with providers for recent network file systems like Dropbox. The VFS 2.0 library allows programmers to access local and network files in a way that is independent of the underlying file system. The standard VFS 2.0 library comes with support for FTP, SFTP, and other traditional file systems which can now be easily extended by the brand new VFSLib.

The new features added by the VFSLib Java library

• Support for recent file systems like Dropbox
• Efficient managed connection pool for stateful protocols
• New VFSConnection class to model VFS connections the easy way
  
• New VFSFileSystemManager class for convenient connection management
• New VFSUtils class with various utility methods (thou shalt not reinvent the wheel)

Please read the following steps to use the VFSLib Java library. This example demonstrates the usage of the Dropbox provider included in the VFSLib ZIP/TGZ archives.

• Step 1: Download VFSLib ZIP/TGZ
• Step 2: Running The Example
• Step 3: The Example Step By Step
  
  • List a Folder
  • Create a Folder
  • Upload a File, Get File Type, Last Modified, And Size
  • Rename a File
  • Download a File
  • Remove a File And Folder
  • Release connection from pool

Step 1: Download VFSLib ZIP/TGZ

First of all please download the current VFSLib ZIP/TGZ archive on your computer. Choose the archive which can be best extracted from your computer (e.g. ZIP for Windows, TGZ for UNIX, ...). The ZIP archive may be unzipped directly from Ant build scripts using the unzip task into a local directory of your choice. Of course you can extract the VFSLib ZIP archive manually with WinZIP or other ZIP extraction tools.

The listing below shows the contents of the current VFSLib archives (list shows relevant files/folders only, library versions may change for newer VFSLib versions):

    api/
    api/index.html
    example/
    example/build-examples.xml
    example/com/lf/vfslib/test/dropbox/ExampleDropboxOptimized.java
    extlib/
    extlib/commons-logging-1.1.jar
    extlib/commons-net-3.3.jar
    extlib/dropbox-core-sdk-1.7.6.jar
    extlib/jackson-core-2.2.3.jar
    extlib/jaxen-1.1.jar
    extlib/jdom-1.0.jar
    extlib/jsch-0.1.44.jar
    extlib/vfs-2.0.jar
    lib/
    lib/vfslib-1.6.1.jar
    lib/vfslib-eng_US-1.6.1.jar
    lib/vfslib-ger_DE-1.6.1.jar
    license/
    README.html
    release_notes.html

The ZIP archive contains various files and resources for VFSLib. The API (Javadoc) can be displayed by opening the index.html file with your favorite Internet browser. In the example folder you can find the precompiled Dropbox example code to be executed directly or using Apache Ant.

The extlib folder contains the additional libraries needed to run the VFSLib properly. For this example you will have to add all libraries to the classpath. The lib folder contains the VFSLib libraries including some language packs (English and German at the moment). The English language pack is mandatory and must be included always along with the main VFSLib JAR file. The other language packs are only needed if your application supports these languages.

In the license folder you can find the VFSLib license as well as the licenses for the 3rd party JAR files residing in extlib. Please take a look at the README.html as a condensed overview of the package. The current release notes for the package can be found in the release_notes.html file.

A compatible Java Runtime Environment (JRE) must be installed prior to the installation of Apache Ant. Most developers can use their normal development environment here. Some IDEs like Netbeans which include a compatible JRE and Ant installation may support the direct execution of Ant build scripts.

Step 2: Running The Example

The VFSLib provider for Dropbox example can be easily executed using Apache Ant. If not already installed please download and install a recent Ant for your operating system. Then you should add the Ant installation folder with the Ant executables to your local classpath so that the ant command may be executed directly from the extracted ZIP/TGZ archive folder (see secion Download VFSLib ZIP/TGZ).

Please open a command line (e.g. MS-DOS under Windows) and type the following commands:

    cd [VFSLib installation folder]\example

    ant -file build-examples.xml run-dropbox-optimized

As figure 1 shows a dialog is being displayed then where you can specify the (evaluation) license for VFSLib. If the license is not valid the example class exits with a message. If you specified a valid license the example proceeds with showing the next dialogs.

Figure 1: Dropbox example license chooser dialog

Figure 2: Dropbox example application key dialog

Figure 3: Dropbox example application secret dialog

You are next asked to specify key and secret for your application. Please go create a Dropbox account and create an application on the Dropbox website. The key and secret generated by Dropbox can be used to fill in the dialog input fields (see figures 2 and 3).

If you specified an evaluation license for VFSLib now an additional dialog will show up where you have to press the "Continue" button to move on. If you have obtained a commercial license already this dialog does not appear.

Figure 4: Dropbox example access token/code

The next dialog is displayed (see figure 4) where you can enter the code (access token) from Dropbox. Simultaneously the example opens your favorite Internet browser showing the Dropbox website to get this code. Remember that it may be necessary to log in to Dropbox first. The access token is necessary to allow your application to access your  Dropbox account.

If all input has been done the example will continue in the background by performing some simple file operations using the specified Dropbox account. Please check out the output of the example script.

If you already have an application key and secret as well as a working access token/code from Dropbox feel free to edit the build-examples.xml file and insert your values. You will have to uncomment the arg tag line. The input dialogs will not be displayed any longer.

Step 3: The Example Step By Step

Let's take a closer look at the Dropbox example implementation. The first thing to do is to configure the VFSLib class which is the central class of the VFSLib library:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

 
// Setup the main VFSLib instance, get evaluation license key from Leisenfels website
VFSLib vfslib = new VFSLib();
vfslib.setLicenseFile(licensefile); // Import from file
 
// We use an optimized file system manager here
VFSFileSystemManager vfsmanager = new VFSFileSystemManager();
 
// Add Dropbox provider to VFS
DbxFileProvider provider = vfslib.addProviderDropbox(vfsmanager, appkey, appsecret, 
 clientidentifier, language);
String scheme = vfslib.getSchemeDropbox();
 
// Pass access token over to VFSLib to access the Dropbox account
FileSystemOptions options = new FileSystemOptions();
DbxFileSystemConfigBuilder builder = new DbxFileSystemConfigBuilder();
builder.setAccessToken(options, token);
builder.setChunkSize(options, 4096); // Allow data portions being transferred
 
// Setup proper account name, the user name for Dropbox URLs (default: anonymous)
// dropbox://[name]:[token]@dropbox.com
String username = "johndoe";
builder.setAccountDisplayName(options, username);
 
// Configure a VFSConnection based on these parameters (optimization, non-standard VFS)
Configuration config = new Configuration();
config.setObject(VFSConnection.VAR_PROTOCOL, scheme);
config.setObject(VFSConnection.PARAM_HOST, "dropbox.com");
config.setObject(VFSConnection.PARAM_USER, username);
config.setObject(VFSConnection.PARAM_USER_PASSWORD, token);
VFSConnection vfsconn = new VFSConnection(vfsmanager, "myconn", config, options);
vfsmanager.addConnection(vfsconn);
 

Configuration of the VFSLib (see ExampleDropboxOptimized.java)

List a Folder

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

 
String uri = scheme + "://" + username + "@dropbox.com";
try {
 FileObject fileobj = vfsmanager.resolveFile(uri); // Determines best VFSConnection to use
 if (VFSUtils.isDirectory(fileobj, false)) { // Non-standard VFS
 FileObject[] childs = fileobj.getChildren();
 for (FileObject next : childs) {
 if (VFSUtils.isDirectory(next, false)) { // Do not print the cleartext token
 System.out.println(" DIR: " + VFSUtils.getDisplayURIObfuscated(next, vfsconn));
 }
 else System.out.println(" FILE: " + VFSUtils.getDisplayURIObfuscated(next, vfsconn));
 }
 }
 else System.out.println(" Entry " + uri + " is not a folder");
}
catch (Exception e) { e.printStackTrace(); }
 

List a folder (see ExampleDropboxOptimized.java)

Create a Folder

1
2
3
4
5
6
7
8
9
10
11
12

 
String tempfolder = scheme + "://" + username + "@dropbox.com/0123456789abcdefghijklmnopqrstuvwxyz";
boolean success = false;
try {
 FileObject fileobj = vfsmanager.resolveFile(tempfolder);
 if (!VFSUtils.exists(fileobj, false)) { // Non-standard VFS
 fileobj.createFolder();
 success = fileobj.exists();
 }
}
catch (Exception e) { e.printStackTrace(); }
 

Create a folder (see ExampleDropboxOptimized.java)

Upload a File, Get File Type, Last Modified, And Size

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

 
String content = "Hello world!";
String encoding = "ISO-8859-1";
 
// Upload a file to Dropbox. Get file type, last modified, and size.
String tempfile = tempfolder + "/readme.txt";
try {
 FileObject fileobj = fsmanager.resolveFile(tempfile);
 fileobj.getContent().setAttribute(VFSLibConstants.ATTR_CONTENT_LENGTH, content.getBytes(encoding).length);
 OutputStream ostream = fileobj.getContent().getOutputStream();
 ostream.write(content.getBytes(encoding));
 ostream.flush();
 ostream.close();
 
 success = fileobj.exists();
 
 System.out.println(" TYPE: " + fileobj.getType().getName());
 System.out.println(" MOD: " + new Date(fileobj.getContent().getLastModifiedTime()));
 System.out.println(" SIZE: " + fileobj.getContent().getSize());
}
catch (Exception e) { e.printStackTrace(); }
 

Upload a file, get file type, last modified, and size (see ExampleDropboxOptimized.java)

Rename a File

1
2
3
4
5
6
7
8
9
10
11

 
String tempfilerenamed = tempfolder + "/README";
try {
 FileObject fileobj = fsmanager.resolveFile(tempfile);
 FileObject fileobjrenamed = fsmanager.resolveFile(tempfilerenamed);
 fileobj.moveTo(fileobjrenamed);
 success = fileobjrenamed.exists();
 if (success) tempfile = tempfilerenamed;
}
catch (Exception e) { e.printStackTrace(); }
 

Rename a file (see ExampleDropboxOptimized.java)

Download a File

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

 
try {
 FileObject fileobj = fsmanager.resolveFile(tempfile);
 ByteArrayOutputStream bostream = new ByteArrayOutputStream();
 InputStream istream = fileobj.getContent().getInputStream();
 int len;
 byte[] buffer = new byte[1024];
 while ((len = istream.read(buffer)) != -1) {
 bostream.write(buffer, 0, len);
 }
 istream.close();
 bostream.flush();
 bostream.close();
 String loaded = new String(bostream.toByteArray(), encoding);
 
 success = loaded.equals(content);
}
catch (Exception e) { e.printStackTrace(); }
 

Download a file (see ExampleDropboxOptimized.java)

Remove a File And Folder

1
2
3
4
5
6
7
8
9
10
11
12

 
try {
 // Remove file
 FileObject fileobj = fsmanager.resolveFile(tempfile);
 success = fileobj.delete();
 
 // Remove folder
 fileobj = fsmanager.resolveFile(tempfolder);
 success = fileobj.delete();
}
catch (Exception e) { e.printStackTrace(); }
 

Remove a file and folder (see ExampleDropboxOptimized.java)

Release connection from pool

1
2
3
4
5
6
7
8
9

 
 try {
 ClientPool pool = ClientPool.getSharedInstance();
 System.out.println(pool.debugPrint());
 pool.releaseForced(); // All connections
 System.out.println(pool.debugPrint());
 }
 catch (Exception e) { e.printStackTrace(); }
 

Release connection from pool (see ExampleDropboxOptimized.java)

How to continue

• VFSLib Java Library
• VFSLib for Dropbox
• Download the current VFSLib archives
• Howto: Receiving Faxes With efax
• Do you have any questions? Please send us a support request.
  

Print Email