My Java Tricks
Get a cup of joe and dive in!
Home     XA File Access in Java     NIO Performance Tips     i18n Files     Logging     Custom Serialization      
 

Transactional File System in Java 

Search the net. You will find a few implementations. I picked Apache commons-transactions. Why this one? Well, traditionally, software comming from the apache foundation is good. I am sure I do not need to mention all the good stuff that came from there. Usually, there is reasonable documentation with software comming from Apache, but surprisingly, not so for commons-transactions. On this page I will show a simple way to perform basic file operations in a transactional manner.

Download and Install Jakarta Commons-Transaction

As the first step get the jars from apache website. You can find binaries here: Download the archive and unpack it on your file system. To work with commons-transaction-1.2.jar you will need commons-transaction jar in the classpath as well as commons-codec-1.2.jar, commons-loging-1.1.jar, geronimo-jta-1.0.1B_spec-1.1.jar and log4j-1.2.8.jar. What are these files? Here is the explanation:

  • commons-transaction-1.2.jar is the main jar. It contains classes that we will use in the examples here.
  • commons-logging and log4j are there to provide loggers.
  • geronimo-jta-1.0.1B_spec-1.1.jar is the JTA (Java Transaction API) spec. You can replace this jar if you want to with JTA spec from SUN. Don't get too excited, commons-transaction is not fully JTA complient. It does not implemented XAResource, or UserTransaction...
  • Finally codec is there for encoding (if necessary).

 

Initialize FileResourceManager


    FileResourceManager frm = new FileResourceManager(storeDir, workDir, false, sLogger);

Let's look at the parameters for the FileResourceMansger:

  • storeDir String - JavaDoc says: "directory where main data should go after commit". This does not have to be an absolute path to the directory where data will go after commit. Instead, this can be a "root" directory for your transaction. For example, if you want to move the file from: /home/joe/data/source to /home/joe/data/destination directory, then pass /home/joe/data as your "storeDir".
  • workDir String - JavaDoc says: directory where transactions store temporary data. This is exactly what JavaDoc says. Transaction log will go into this temporary directory. In our example, this could be: /home/joe/tx_tmp directory.
  • next parameter is a boolean that indicates if the path should be URL encoded. In our example it is false, indicating that we have simple path with no spaces or i18n characters in it.
  • sLogger LoggerFacade - JavaDoc says: the logger to be used by this store. Interestingly enough, this API requires a logger. So, to use commons-transactions you must have logging :) No "System.out" stuff... I guess, due to the nature of the operation (i.e. transaction) decision was made to force users to supply logger.


Now that you have FileResourceManager object, start it:

    frm.start();

 

When you start the FileResourceManager, it will initialize its internal state, and then it will look for any transaction that might not have completed. This could happen if, for example, JVM crashes in the middle of the transaction. Transaction status will be examined and unless it was equal to "STATUS_COMMITING", FileResourceManager will attempt to rollback your failed transaction. If the recovery is not succesfull, transaction will be marked as "dirty" requiring manual intervention. This behavior is very similar to the transactional behavior of the traditional database.

Start the Transaction

Every transaction is associated with the unique Id. To start the transaction, you must supply this identifier. If you do not want to create your own, you can ask FileResourceManager to generate one for you.

    String txId = frm.generateUniqueTxId();

Now, that you have the id, you can start the transaction:

frm.startTransaction(txId);

Modify Resources

With the transaction started, you can begin file manipulation. You can perform following operations:

  • copyResource
  • createResource
  • deleteResource
  • moveResource
  • writeResource


Here is an example that would delete one file and move another:

    try {
      frm.deleteResource(txId, "deleteMe.txt");
      frm.moveResource(txId, "fromFile.txt", "destinationFile.txt", true);
      frm.prepareTransaction(txId);
      frm.commitTransaction(txId);
    } catch (ResourceManagerException e) {
       e.printStackTrace();
       try {
          frm.rollbackTransaction(txId);
       } catch (Exception e1) {
            e1.printStackTrace();
      }
   }



Note that resource ids in the example above are relative to the "storeDir" as initialized when FileResourceManager was constructed....

 

Recovering Failed Transactions

 

This is the interesting part.... There are several use-cases that should be considered for recovering scenario, but the recovery is very similar. When the FileResourceManager starts it will examine working directory for any transactions that might have not been completed in the previous run. If any transactions are discovered, transaction recovery starts. Depending on where the transaction was when the system crashed auto-recovery might try to rollback or roll-foward the transaction. Roll-forward will be selected only if the transaction was already in the process of commiting when system crashed or when it encountered unrecoverable problem.  If the rollback, or roll-forward is successful, FileResourceManager will be initialized and new transactions can proceed.

However, if the transaction cannot be recovered, due to the (for example) I/O problem, FileResourceManager will mark the whole database (i.e. your working directory) as "dirty" and will not allow any more changes to the database until issue is resolved.

 

How to clean "dirty" database?

Cleanup is usually manual operation.

First step is to understand what went wrong. Before database is marked as "dirty" there will be an exception that is traced using a logger you had to provide when constructing FileResourceManager. So, check your logs. Find out the root cause for the issue. Once you understand what went wrong, fix it. For example, if file could not be deleted because of the permissions, change the permissions to allow file deletion, then invoke "recover" method on the FileResourceManager.

If you cannot figure out what went wrong, but want to allow further processing, you can "reset" the database. Just be VERY carefull here. Calling default "reset" method will REMOVE store and work directory. I am not quite sure why was the decision made to remove store directory during reset. In my opinion that is very dangerous and not something you would ever want to do in production environment. Instead I suggest you extend FileResourceManager and override "reset" method. Code that is performing the "reset" would then just move the current work directory to a different name/place to be resolved later and create new "clean" work directory. Once reset is performed, you must restart FileResourceManager to initiate clean start and continue processing transaction.

Conclusion

If you have to operate on files and cannot afford to loose data, seriously consider using commons-transactions package. It is probably better then any custom mechanism you can come up with. It supports 2 phase commit (as indicated in the example in "Modify Resource" section). With a little work, you could probably make it into an XA complient resource.



Send any comments to: admin@myjavatricks.com