Advanced Analytics

Datafication refers to the fact that we are looking at more and more things and using technology to render them into a data format. Simply said, it is about taking previously invisible process/activity and turning it into data, that can be monitored, tracked, analyzed and optimised through analytics. Whereas digitization has been a process taking place over many decades now, datafication is a relatively new phenomenon. The difference being that whereas digitization was about converting information into a digital format, datafication is more about the interaction between the digital domain and physical objects, processes, and environments

Big data is a term that has come to be used in reference to data structures that are diverse, complex, and of a massive scale. Although the term has been in use since the 1990s it is only with the rise of web 2.0, mobile computing and the internet of things that organizations find themselves increasingly faced with a new scale and complexity of data. The term big data implies an increase in the quantity of data, but it also results in a qualitative transformation of how we store and analyze such data - it is certainly the case with big data that more is different.

This explosion of data is of course far outstripping our capacities to use it. A small fraction is in a traditional structure form that is easily accessible and usable by organizations, a larger section of big data is unstructured but at least somewhat accessible, while the vast majority is simply hidden all together going unseen and unused, this, we can call dark data. Data may be considered dark for a number of different reasons because it is unstructured, because it is behind a firewall on the internet, it may be dark because of speed or volume, or because people simply have not made the connections between the different datasets.

In many organizations, large collections of both structured and unstructured data sit idle. On the structured side, it’s typically because connections haven’t been easy to make between disparate data sets that may have meaning—especially information that lives outside of a given system, business unit or function.

Advanced analytics brings about a more objective form of decision making, what is called data-driven decision making. The implicit premise of big data is that decisions can be made wholly based upon data and computerized models, shifting the locus of decision making from people and intuition to data and formal models.

Dataism may be recognized as the general underlying philosophy of big data which holds data as a primary source of truth in its own right. The basic premise is that because Big Data can capture a whole domain, providing a complete high-resolution dataset, there is no need for prior theory, models or hypotheses, as through the application of data analytics the data can speak for themselves free of human bias or framing. Meaning transcends context or domain-specific knowledge and is thus neutral being able to be interpreted by anyone.

With information technology, we have gone from a world that was private by default, to a world that is public by default. In a pre-digital world, our lives were primarily private by default, by the fact that most of our communications were not mediated by tools for mass communications; that our conversations were bounded by the physical location and thus it took extra resources and effort to publish publicly. Now it takes extra effort to make it private. This massive amount of data that is being generated by people can, of course, be used for beneficial outcomes or for detrimental outcomes. This social data can be used by researchers to understand society better, it can be used for beneficial security reasons, it can be used to enhance services provided. But there are growing concerns that our data is or might be used against us in a multiplicity of ways.

The term algorithm is currently making a meteoric rise to fame. A geeky term that was previously confined to the world of mathematicians and software engineers is making its way into the mainstream, as people are increasingly recognizing the material impact on society that algorithms are starting to have. Algorithms, that used to be buried away inside of computer program files, used to find the derivative of a slope, or to find the shortest path between two locations, have today expanded to almost all areas of human activity.

Machine learning refers to the process through which a computer can construct an algorithm based upon the analysis of data. Such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions, through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance are difficult or infeasible. In such cases we tell the computer what we want the output to be and then it builds the model based upon the data that will be able to produce those results when presented with new data sources to process.

Machine learning is a challenging area of computer science and engineering and there are many different approaches to building machine learning systems. As yet there is no formal classification of these different approaches but in his book The Master Algorithm, Pedro Domingos of the University of Washington provides a coherent and accessible overview to the different methods currently being pursued. In this video we give an outline to the five different approaches described in the book.

Artificial neural networks are computing systems inspired by the biological neural network of the brain. Such systems can progressively improve their ability to do tasks and recognize patterns by learning from examples. Artificial neural networks are in their essence computational networks that can perform certain specific tasks like clustering, classification, pattern recognition. They do this by representing patterns in data as networks of connections between nodes on the network. They then learn by altering the strength of the connections between the nodes to create new network structures that can represent new patterns.

Deep nets are the current state of the art in pattern recognition, but they build upon a decades-old technology of neural networks talked about in the past module. Deep learning is a machine learning method based on neural networks. What distinguishes deep learning from the more general approach of neural networks is its use of multiple layers within the network to represent different levels of abstraction. Deep learning algorithms use a cascading structure with multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. In this way, they learn multiple levels of representation that correlate to different levels of abstraction.

For better or worse our world is in the midst of a silent algorithmic revolution. Many of the decisions that humans once made are being handed over to mathematical formulas. Today, we expect algorithms to provide us with the answer—who to date, where to live, how to deal with an economic problem. With the correct algorithms, the idea is, computers can drive cars better than human drivers, trade stocks better than Wall Street traders and deliver to us the news we want to read better than newspaper publishers. In this video, we talk about what algorithms can and can't do.

These smart systems incorporate functions of sensing, actuation, and control in order to describe and analyze a situation and make decisions based on the available data in a predictive or adaptive manner, thereby performing smart actions. In most cases, the “smartness” of the system can be traced back to autonomous operations based on closed loop control, machine learning and networking capabilities that enable the system to exhibit adaptive behavior. These smart systems will sit at the intersection of humans and our technology infrastructure as they perform basic control operations for our technology infrastructure and interact with people so as to understand their needs and perform required actions.

The platform model will be important in developing smart solutions in that it will enable different smart capabilities to be offered as modular utility functions that can then be plugged into and bundled together by enterprises according to their specific needs. Instead of having just one general purpose system, a platform model allows developers to draw upon specific capabilities and integrate them into their solutions, such as machine learning to recognize a face, or voice recognition software, or advanced analytics for specific domains. Equally the platform, plug and play model will work to commoditize smart systems making them available as a service to almost any technology developer. APIs and developer toolkits are already offered by IBM for their system Watson that can be plugged into a wide variety of applications from health diagnostics to analyzing data coming from transport systems. In such a way smart capabilities will flow to almost all types of technologies in the coming decades.