Getting Agile

Harry Pierson points out in his blog entry, “The SQL Complexity Problem”, the difficulties with relational database operations and maintenance. The relational database has been a great tool for expanding business technology into the information age. With this advancement we have found the limitations of relational databases whether it be the ease of implementation or relation models themselves. The relational database is still essential in day-to-day persistence but is it the best choice for enabling fluid data throughout your organization, consolidating disperate data sets, driving information to distributed enterprise data centers, and other 21st century IT data needs?

My current thought on this is essentially “no“ with a small caveat, there is still a place for relational databases when persisting data for historical record and reporting is a must. The problem I have with relational databases in the data access domain are the hoops which teams must jump through in order to make their “layers” play nice with each other. This includes persisting all data along with any type relations, flags, and reference data. These extraneous pieces of data bloat our databases, create maintenance headaches and inhibit agility in our businesses. What if we no longer made the relational database our gateway to corporate data? What if we allowed our corporate data to associate with other data based upon the context?

There are some great existing technologies out there which may be a precursor to such a revolution. Topic maps is a simple idea about representing data through topics, associations and occurences. Much of this is explained in “An Introduction to Topic Maps” by Kal Ahmed and Graham Moore but I will give a brief overview here. An association is essentially a grouping of topics which are themselves computer representations of a concept. Occurences are informational resources about a topic such as a description or an address to the informational resource. The relations between topics are simple, decoupled and based on context. One of the most interesting concepts to me are the heuristics within the topic maps standard to identify topics which represent the same subject.

I have written before about my issues with our current relational addiction. I like the question Harry posed at the end of his entry, “What other ideas would make persistant data significantly easier to work with?”. If we can answer this question we may find less resource intensive and more cost effective ways of paying off our technological debt as opposed to the language parading and disputing which I think are only distracting us from the real problems occuring today.