Category Archives: Distributed Computing

Top 25 Open Source Projects — Recommended for Enterprise Use

This is a bit off my usual topics on this blog but I am a heavy open source user and this article is something that I hope gets to more enterprise operations, managers and executives. I have been using and deploying production available applications using open source tools, libraries, and platforms for over 12 years now. Open source tools can do almost anything commercial products are able to do and have transformed the software industry in that time span. The list given in the article contains open source projects that I would recommend and have used in the past either directly or indirectly including *nix tools and libraries shown.

I would like to add to this listing with some of the tools I have come to use often:

  • Maven 2.x+ (http://maven.apache.org/)
  • JBoss (http://www.jboss.org/)
  • Rio/Jini/Apache River (http://incubator.apache.org/river/RIVER/index.html)
  • Apache Commons (http://commons.apache.org/)
  • Subversion (http://subversion.tigris.org/)
  • Apache Web Server (http://httpd.apache.org/)
  • Bouncy Castle (http://www.bouncycastle.org/)
  • Time and Money (http://timeandmoney.sourceforge.net/)
  • Spring Framework (http://www.springframework.org/)
  • Hadoop (http://hadoop.apache.org/)
  • Ruby on Rails (http://www.rubyonrails.org/)

This is some of the open source that I have and still use on my projects. What are your favorites that were not on the list?

Mobile Code and AJAX

Mobile code is an essential programming model for the current connected user paradigm which has been deemed “Web 2.0”. There are many examples of mobile code including Java RMI proxies (Jini/JERI smart proxies), browser downloaded JavaScript, Java applets and ActiveX controls. Sometimes mobile code is overlooked in the design of enterprise applications as a medium to increase application usability, scalability, and awareness. For example, the power of Jini is really it‘s ability to locate services from anywhere on the LAN and communicate using a protocol which is defined by the remote service. In fact, services may include all of the functionality inside the downloaded service proxy and therefore eliminate the need to communicate back to the remote service.

In terms of RIA (Rich Internet Applications), the use of XML and/or JSON communication from downloaded Flash and/or JavaScript allows for asynchronous calls to the server for data updates to the UI. This ability to update data without reloading the page becomes an important function of web application usability by giving better user feedback and lower bandwidth overhead. The use of XML as a message format for this communication also enables standard integration into enterprise and consumer services which are usually contacted with REST or SOAP messages. A term has even been developed to describe this type of communication, “Client/SOA”.

My personal bias for web applications with AJAX recently has been using DWR (Direct Web Remoting) to expose POJO (plain old Java object) as JavaScript objects. The reason that I have adopted this approach to my AJAX development is DWR‘s ease of use and the separation of concerns. This separation is orchestrated by allowing web designers to develop the user interface with less concern for their integration with server side development. Here is an example of this separation with DWR:

<html>
<head>
...
<script language="text/javascript" src="/dwr/engine.js">
<script language="text/javascript" src="/dwr/util.js">
<script language="text/javascript" src="/dwr/MyPojo.js">
<script language="text/javascript">
function getFooFromPojo() {
MyPojo.getFoo('setFooValue'); // send callback function name
}
function setFooValue(value) {
DWRUtil.setValue('myDivForFooValue', value); // find DIV by ID to put value into
}
</script>
...
</head>
<body onload="getFooFromPojo();">
...
<div id="myDivForFooValue'>
</div>
...
</body>
</html>

The HTML and JavaScript is quite simple in this example. We have imported the DWR specific JavaScript sources, engine.js and util.js, along with my exposed POJO in MyPojo.js. The MyPojo.js script is generated using some DWR magic which I will show later. The ‘‘getFooFromPojo()‘‘ function calls on the MyPojo Java class method ‘‘getFoo()‘‘. The JavaScript version of the MyPojo ‘‘getFoo()‘‘ method also takes the callback function for the resulting value to be handled by as the last argument. In this case we sent the ‘‘setFooValue‘‘ function as the callback function with the resulting value as a parameter. Inside the callback function we set the text of a DIV with ID ‘‘myDivForFooValue‘‘ to the resulting value.

Now to show the magic. DWR exposes POJO method ‘‘getFoo()‘‘ through an XML descriptor by default named dwr.xml and placed in your [WAR (web application resource)|http://access1.sun.com/techarticles/simple.WAR.html] under the ‘‘WEB-INF/‘‘ folder. An example of the content within this file looks like this:

<!DOCTYPE dwr PUBLIC
"-//GetAhead Limited//DTD Direct Web Remoting 1.0//EN"
"http://www.getahead.ltd.uk/dwr/dwr10.dtd">

<dwr>
<allow>
<create creator="com.acme.example.MyPojo" javascript="MyPojo"/>
</allow>
</dwr>

The basic description of what is contained in this example dwr.xml file is that the class ‘‘com.acme.example.MyPojo‘‘ will be available as a JavaScript object by the name of ‘‘MyPojo‘‘. In this case all of the public functions of MyPojo, besides some of the ‘‘java.lang.Object‘‘ base functions, are available through the JavaScript equivalent object. My class may look something like this:

package com.acme.example;

public class MyPojo {

public String getFoo() {
return FooRepository.firstFoo();
}

}

This simple example may be enhanced to use other JavaScript sources and web services within the same page. This mashup of multiple services (exposed POJOs, web services, JavaScript functions, etc…) will produce an ever expanding realm of user interaction possibilities. An example of this may be an enterprise application which contains data such as customer location. You may create a mashup of this data with Google Maps to display the customer‘s location visually on a page with driving directions.

The examples above are only one of the many examples for mobile code. A large interest of my own is intelligent agents and peer-to-peer systems. There are multiple technologies which fit into this market (Jini, JXTA, SUMO (Suggested Upper Merged Ontology), OAA (Open Agent Architecture), etc…) which are not described in this blog entry. These technologies may enhance the user experience through dynamic service interaction, context awareness, and a suggested language for communication between users and services. It is an exciting time in which innovation is again a part of the technology domain. Mobile code will enable a whole new user experience as we move past web 2.0.

P2P Makes Cents

Robert X. Cringely of “I, Cringely” fame brought up the world of P2P commercial ventures in “Why P2P Is the Future of Media Distribution Even If ISPs Have Yet to Figure That Out”. I am long time fan of P2P with a dash of service-orientation from the stance of JXTA and Jini. The assertions put forth in this article point to our highly substandard bandwidth pipe which will need to be increased for the upcoming Internet media age. It mentions a few companies, two of which, Grid Networks and Peer Impact, base their network client infrastructure around grid technologies much like BitTorrent.

Cringely makes a very good case for the adoption of peer-to-peer client networking for ISPs. This will allow ISPs to maximize their bandwidth usage and resell content directly to their customers for our up and coming Internet media providers. Looks like an interesting cast of vendors out there ready for the opportunity to provide media services to the broadest audience possible. It was not so long ago that I worked for an IPv6 network backbone, Zama Networks, which had aspirations of providing VoIP transport between their data centers around the Pacific Rim. We had a beautiful data center, a rich set of network metrics, and a very advanced IPv6 knowledge base for our customers. If only they had lived long enough to participate in solving the networking issues of today.

SOA is for Kids

Not that I wish to be apologetic regarding the title of this entry, but I feel it needs some explanation. The title by no means is intended to suggest SOA (Service Oriented Architecture) is simple or without merit. The intent of this entry is to assert the importance of SOA as a means to improve enterprise architectures and their extended maintenance complexity. Given this assertion, I contend that SOA must be a critical part of computer science programs and corporate training strategies for software development. Also, SOA is extremely complimentary to agile methodologies in the delivery of high quality and valuable software assets in small calculated chunks.

The rise of layered architectures, such as those presented in J2EE blueprints and MVC (Model View Controller) based frameworks like Spring and Struts, has been integral to a design which contains a middle layer for business logic. This layer, also known as the “service layer”, provides valuable support and accelerates delivery of business products and services. A problem which has persisted with these architectures is their lack of complete service decoupling. Teams must combat coupled designs through slices of the layered architecture with pre-defined interface documentation, technical requirements documents, coding standards, and code reviews.

In my opinion, the frameworks mentioned above are not at fault. I believe that the problem of service coupling is due to our training techniques for developer application design over the past few years. A shift in software development design must be made such as those made when structured, function prototype, and object-oriented design were introduced. I should mention a friend of mine who has wrote a white paper, “A View-Oriented Approach to SOA” by Scott Came, which attempts to define an appropriate method for SOA design. I agree with many of the points made in his white paper and would like to describe effective programming models which facilitate good SOA design.

The following image depicts the high level systems within an SOA implementation:

SOA Overview

Each component within the SOA has a role in the effective design:

  • Service Registry – This is a location where clients may find services they are interested in using. Examples of a service registry are UDDI, Jini's Reggie, and Spring XML configuration descriptors.
  • Service Proxy – This is the interface defined for communicating with your service. This may be a compiled WSDL to Java or C#. The proxy may be as smart or dumb as your service requires. RMI uses downloadable serialized proxy objects to communicate with a remote service. For Spring you use the AOP proxy object created at runtime using an XML configuration file.
  • Deployment Configuration – A deployment configuration allows service to be used dynamically at runtime. In Jini you might configure your service to use the start.jar with net.jini.config.ConfigurationFile instance which defines the communication protocol to use between your service proxy and the service implementation. Another example is the J2EE application descriptors such as those used in EJB configuration and JNDI name binding.
  • Message – The data sent to your service for processing such as XML for SOAP or REST web services, Java objects for RMI, or JMS messages into a queue.
  • Service Grid / Bus – In many distributed systems, containers are used to manage objects across a grid or bus. J2EE defines an EJB container for managing services and entities. Rio is a grid fabric which is built on top of Jini which manages JSB (Jini Service Beans). ESB (Enterprise Service Bus) usually have service containers which have been standardized in the Java community through the JBI (Java Business Integration) specification.
  • Service Interface / Implementation – The service interface defines how clients may communicate messages with a service. This can be defined using WSDL, Java interfaces or POJO (plain old Java objects), or IDL in CORBA. The implementation of this service is the message handler.

In my experience, there is an initial barrier to entry for SOA development which seems obvious at first glance but derails many developers, including myself at one time or another. When creating a desktop or console application, most of the time we are working with in-memory models which can be passed around in graphs easily between components. SOA-based applications or systems revolve more around message passing between clients and services.

The difference in paradigm brings some different design philosophies such as the amount of data and the frequency we communicate between components or services of the system. We might not think twice about making a call for each item added to our order if it is inside our desktop application. If we are adding these items to our order over the network you may fall pray to “The Eight Fallacies of Distributed Computing”. A possible way to combat this is a local persistent cache and guaranteed delivery options such as those found in JMS.

Many times, SOA-based applications or systems are a step away from overly evolved OO designs. Since a service tends to contain finer grained slices of functionality, developers can focus more of their efforts on handling the task at hand. For example, adding an item to an order may only entail finding the order by a unique identifier and then adding the specific item passed into the service via a message. In a service grid or bus there may be any number of services which can perform this action. And all of those services may just be copies of each other and therefore handle the message in the same manner.

To be continued…

Data Management for the Future

I have been rambling to colleagues about how much I despise relational databases for many reasons. These include the hierarchical formation of data, the finite set of relationships between tables, their inability to distribute themselves, and the maintenance overhead that they create for application and integration development projects.

One approach that I thought was a very viable alternative to the relational database is a grid of services with the ability to persist only those data elements which are worth the effort. I would contend that a large amount of current data sets do not need to be persisted into a relational database and are either temporary or reference data. Persisting this type of data becomes a hindrance to maintenance objectives as a database grows over time and possibly, or even probably, after an initial release.

I happened to have a magazine laying around that I got at the Java One Conference this year. It was the Dr. Dobbs Journal which had an article called High-Performance Data Management with Java inside. This was quite a coincidence since I had just come back from a lunch conversation with some colleagues where I was trying to communicate my distaste for relational databases and the use of grids for data access as an alternative. I thought that the article was a great overview of this approach to data management. The sub title for the article I thought described the high level parts quite well:

On-disk persistent storage, in-memory data storage, & cache management

If you get a chance, please take a look at the article. I hope to see a revolution in the data persistence layer of our future applications which will remove some shortcomings of the current relational hell.