Technology | Why Big Data is a big deal for law firms

The inexorable rise of cloud computing and the ever increasing need for data storage has provided the technology world with a new problem to deal with. The term big data refers to the storage of vast quantities of computer-generated information in ever increasingly large and complex silos.

Of course it was not always like this. The history of data storage started with the likes of cardboard punch cards, storing patterns for early computers to undertake repeated tasks automatically.

Magnetic data storage was a huge leap forward for the industry, allowing computer equipment to repeatedly write to data storage media as well as read from it. This changed the game completely, and the first database systems were formed, along with early attempts at the engineering and system rules that go with such new inventions.

Over the years, data storage turned in to database theory and practice, and this evolved to allow more complex data structures to be used by ever more complex systems and machinery. Simple flat file databases morphed into relational databases which in turn became object oriented and object relational databases.

Data about data

These changes allowed the likes of IBM, SAP, Oracle and other large system vendors to run data storage facilities that could keep up with the size and complexity of the software they sold, and the volume of transactions that their business customers put through them.

This huge amount of stored information also gave rise to a new phenomenon of firms using data about data. If you consider the amount of data that the likes of Google and Facebook have at their disposal, the ability to categorise, cross reference and pattern analyse it opens up a new world of marketing and information mining that is useful to them, and to those firms they sell services to.

Database technology not only has to cope with an ever increasing quantity of data, it also has to cope with a widening variety. If you go back twenty years, we really only stored text and numbers. Now we expect to store videos, music and pictures among others.

Then there is the sheer number of transactions that a system has to be able to cope with, sometime referred to as the velocity of data. The need to service more people, using a widening variety of assess methods has exponentially increased the number of data transactions. In the background we have also seen the rise of virtualisation of all facets of computing, including data. This added a technically complex layer, and combined with the above has stretched the traditional tools that were designed to manage large data storage functions beyond their ability to do so.

The quantities of data being produced on a daily basis take some digesting. Not that long ago we used to save information to disks that held just over one megabyte (MB). Now phone vendors will sell you a handset with 64 gigabytes of data storage. That equates to 64,000MB, and the chances are that you will easily fill that space. If you use a service such as iCloud, it is conceivable that you could store most of that both on the phone and in the cloud. If you multiply that by the number of iPhone users, you generate numbers that are hard to imagine. And that is just one example.

So Big Data theory, and new tools and techniques to store, manage, retrieve and protect the fidelity of all this information have become a hot topic, and one you will no doubt hear more about.

Challenges for law firms

There are a few things for the legal world to consider.

Firstly, it is likely that more and more firms' data will move over to the kinds of storage functions that Big Data is applicable to. This will happen in a variety of ways. Some firms who have outsourced some or all of their information technology are likely to have their data stored in one of these facilities.

Firms buying services from online suppliers (and this covers both software and information services) will also have a direct or indirect involvement. This means that we are moving away from firms storing their own data. While this is a proven concept, it is still incumbent on organisations to maintain and protect their data, so some knowledge of the storage function will always be required.

Another aspect of this subject is that those firms giving advice on software and data transactions will now have a new knowledge set to ingest and advise on. Those working at the sharp end of computing law will need to pick up on the issues, ramifications, tools and techniques of Big Data in order to keep themselves at the cutting edge. Those involved in privacy and data protection issues have a new a set of concepts to grapple with.

A further twist in the tale for law firms is that of electronic discovery and data mining. As the years roll on, more and more legal cases will need to draw on stored electronic data. The more complex the data mechanism, and indeed the data stored, the more tools and techniques will need to be applied in order to extract what may be required. Data mining has evolved from simple searches to multifaceted exercises where issues such as audit trails, artefacts, disparate locations and storage methods are now part and parcel of that landscape.