Managed File Transfer

File transfer in an enterprise isn't as simple as it sounds. One may go on on about the problems around it, few of the common problems are tracking, monitoring to ensure successful transfer.

Traditionally, File Transfer Protocol (FTP) has been used as a medium for exchanging files between internal and external systems. As the business and security need rises, it is relatively difficult manage such transfers using plain FTP. Nowadays, File transfers are not just limited to FTP but it also happens over FTPS, SSH FTP, OFTP, HTTP and more sophisticated protocols like AS2, Web Services, SOAP, EBXML etc.

The more advanced protocols offers greater security and flexibility in terms of correlation etc. And, this is good for an enterprise/industry to securely transfer files/data over wire. However, the basic problem of tracking, monitoring and error handling remains to be a bottleneck.

We need a system to manage such transfers and that should have capability of auditing, tracking and redelivery mechanism in case of transfer failures. This is the problem "Managed File Transfer" tries to address.

Open source project

Just started an open source project, please have a look and help me to make it more success http://colossaloftpv2.sourceforge.net/

SOA Governance

I am up on a stage to present a session on "SOA Governance for Architects" as part of "Oracle Develop" in Mumbai next month. Here is a quick brief about the session,

"As companies continue the mass migration to SOA, effective governance becomes increasingly important. This session addresses SOA governance from the architect's perspective—aligning applications and services with the reference architecture and ensuring that progress and compliance are tracked and visible. Attendees will learn how to apply governance throughout the SOA lifecycle and will hear about key organizational practices for keeping SOA aligned with architecture and business goals."

More details here: http://www.oracle.com/events/oracledevelop/mumbai/index.html

Added to the presentation, there is a hands-on lab i may be out there to help as well. There is much more on the stall other than SOA in the two days event.

JDK1.6 Hotspot VM leak - Out of swap space

There's a leak in JDK1.6 HotSpot VM and it doesn't appear to be fixed in JDK1.6. I was surprised to see this behavior because things are fine from java heap point of view. After spending quite amount of time debugging the application, figured out the leak is in VM.

In JDK1.6, if there's any "ClassCastException" happens, those are kept in the JVM native layer and those are never get cleared. This would result in "OutOfMemoryError: ... Out of swap space?". There's a quite lot of such exception happens when the application is executing XA transactions, added to my surprise this happens on the underlying infrastructure (j2ee server) so the application is not the one to be blamed though ;).

For the above reason, jvm crashes in less than two hours (depends on the load). Tracked the code share/vm/runtime/sharedRuntime.cpp from the crash log. Things appears to go wrong in the function generate_class_cast_message.

Problem: When there's a "ClassCastException", the exception trace/message has been added to "NEW_C_HEAP_ARRAY" and those messages never gets cleared.

Fix: Just use "NEW_RESOURCE_ARRAY" to hold the exception message, things should be fine.

Build:

One could pick up the source from java.net (please refer the licensing agreement of OpenJDK before making any change, there's JRL one for research and study purpose you may try ...). Follow the instructions provided in the source bundle to build your fixes.

I just set the following in my linux terminal (pre-req. in my linux box) and i am all set to go.

GCC - pre-req: In the prerequisite section at the top, it's mentioned as gcc 4 but i was on gcc version 3.4.6 20060404 and could build VM successfully.

1. ALT_BOOTDIR
2. ALT_BINARY_PLUGS_PATH
3. cd [src]/hotspot/make
4. make

Note: I was interested in building only the VM and not the whole JDK. If you want to build the JDK, just do it from the top level.

Debug:
gdb: is fairly fine for native code debugging.

JAX-RS: RESTful Web Services

Shall be updated soon.

Concurrent processing (threads) - J2EE Applications

Often we come across a question, "How do i process things asynchronously in J2EE application?" - The only answer to this question is MDB. Let's rule out the option of creating unmanaged threads on our own, because those may interrupt with the orderly shutdown of an appserver and several limitations in the context (J2EE). Like many people in the industry, i also like the MDB and feel it's the best among the others in the enterprise beans stack. The question here is, Is this enough to address all the needs of sophisticated applications?

Perhaps, in my opinion, the answer is no. I could give several use cases to back my opinion, think of performance when there's a need for couple of asynchronous process to complete a use case on a heavy load. I could throw some classic use cases, if needed.

I looked at the alternatives and realized there's no standard way to create/or get hold of a managed threads in a J2EE applications to do our business. A sophisticated web application may need to spawn its own threads to complete it's business but in today's J2EE world there's no standard or defined way yet. There are some application server vendors who may provide option for concurrent processing in a proprietary way and some initiatives in JCP to address this also, let's wait and watch the expert group take on this. The portability of application across the container is in question here.

Below is an alternative what i could think of to address this.

Architectural Solution

There's no change in the business components end and it continue to remain same. The approach for asynchronous operations has been changed from MDB to JCA RA (Resource adapter). Some may have a question of "why are we talking about JCA RA here?". The answer is, this is the only way where i could delegate the asynchronous business to J2EE container.

Here're the steps to achieve the above,

1. Create a resource adapter and get the javax.resource.spi.work.WorkManager instance from the bootstrap context (ref: javax.resource.spi.ResourceAdapter) and keep it in the memory that can be accessed by your components (you could think of getter/setter to access the same in a singleton or a similar class).

2. Package the JCA RA along with your existing J2EE application(.ear). You need to make sure your applications in the .ear, share the same classloader.

3. From your application, you could just create a javax.resource.spi.work.Work and submit the same to javax.resource.spi.work.WorkManager for execution. In the work implementation, inside run method you could write your piece of business.

References:
JCA Specification - 1.5

Some may think, how it's going to be better than MDB approach. The answer is, quiet simple, we are not doing any JNDI lookups, send and receive message etc which is not require or mandatory from the application business point of view. Think of concurrent request of say 100, if you could avoid 100*3 (assume three MDB) JNDI lookups, send and receive message activities, i bet there would be a considerable difference in performance. Because, in the above discussed JCA RA approach, you avoid all these and just invoke methods on objects and you still get the benefit of asynchronous behaviour managed by your J2EE container.

Note: The above discussed approach may be worth considering to address the asynchronous needs in a J2EE container until we have something concrete in J2EE space. I presume there are some initiatives in the JCP and JSR has been proposed to address this. Eager, to see what's coming up in J2EE 1.5 space.