One plus one equals three – the power of Data Combinations
The information that can be learnt from data is hidden in its size, its speed and its accuracy. When data sets are combined real insight is revealed.
Data analysis began during World War II when the cryptologists at GCHQ Bletchley Park, England under the leadership of Tommy Flowers built ‘Colossus’; the world’s first programmable electronic computer which was able to break the German military Lorenz cypher by analysing what at the time were massive data sets. Colossus compared two character streams, counting statistics from the evaluation of programmable Boolean functions.
How does Luciad combine data?
Today we can bring together information from real time dynamic data sets unparalleled in size as well as historical information to reveal never before seen results. For example to illustrate the economic as well as human consequences of another major hurricane impacting on the East Coast of the USA, Luciad was used to combine twenty years’ worth of North Atlantic meteorological data, the insurance records of every commercial and domestic property in the continental US, high resolution satellite photo records and the database of the entire road network. This allowed us to run real time ‘what if’ scenarios that precisely show the destructive effects of a storm enabling individual insurance companies to understand their risk exposure on a street by street and house by house basis as well as helping emergency services to understand where best to position manpower and equipment. In addition this same insight can assist the local government and coastal authorities anticipate future tidal protection requirements.
Airlines and aircraft engine manufacturers are coming together and are now using the same technology to undertake analysis of their fleets by combining real time data from every aircraft engine with meteorological, atmospheric, time and location data at various flight levels. The resulting insight shows how the operational stresses on an engine and an airframe vary across the globe depending on flight circumstances and can be used to make predictions of likely component wear or failure and to identify those flight routes that cause the largest number of problems.
The key to successful large data set combination is retaining the source data in its original format to prevent the introduction of errors during transformation and to enable the analysis speed that is required, to quote Luciad CTO Frank Suykens,
“In the past the time taken to analyse data meant that the cost of analysis was too high to be of value.”
5D Digital Reality
Legacy era geospatial systems that are unable to process large data sets without transformation are already obsolete and as we move to a world where we must all be ready to deal with even more sensor developments such as constant real time global satellite video images of the entire planet the requirement for high processing speed of massive data set combinations becomes mission critical. Furthermore as we increase our use of temporal analysis and move into a world of 5D analytics it will be the map that is going to be at the heart of all we do.
Luciad’s visualisation technologies support live connections to dynamic sensor feeds in a 3D environment. The result is a 5D digital reality: real-time, rapid fusion of multi-source content and the ability to perform analytics on-the-fly. These intuitive systems and their ability to combine enormous quantities of data, benefit all kinds of applications - from public safety and smart cities to defence and intelligence - enabling users to make critical decisions based on changing information in real time. Data integration is about managing complexity, streamlining connections, and making it easy to deliver information to any part of the organisation via whatever device is most suitable. Applying location and temporal intelligence to a static dataset makes it spatially comprehensive and offers new levels of insight around that dataset, which leads to increased understanding and comprehension.
Combining multiple Big Data information feeds, maintaining source data formats and class leading processing speed are the secrets to gaining the information insight that will is critical for decision making and ultimately enables us to turn data into information, and information into insight.