In a world drastically and constantly changing, industries and technologies are being transformed at a rate never before experienced. Recently, I had the opportunity to speak with Roy Schulte, Distinguished VP Analyst at Gartner, about trends in the evolving world of big data, advanced analytics, and decision intelligence — trends that are powered by advancements in cloud technology, machine learning, and real-time streaming. Below, I will share a summary of key highlights.
The growth of decision management
More businesses are implementing decision management. They want to make more automated decisions to accelerate outcomes while improving applicability. In tandem, decisions are becoming more complicated with regulators increasingly expecting decisions to be explainable and with audit trails built-in. Equally, the combination of machine learning and decision management has placed greater focus on the importance of avoiding bias.
Bringing all of this together promises continuous decision improvement; updating models and strategies in days or even hours rather than weeks and months. Essential in responding to the rapidly changing environments, none more so than the impact of Covid-19.
As businesses are implementing decision management, they are putting the new systems into the cloud. Based on a Gartner survey in mid-2020, 67% of respondents said their digital business platform will be cloud-native application architecture. It’s the primary criteria for the architecture of many of these new systems.
Migrating to the cloud
The reason decision management is going to the cloud is the same reason other areas of business are taking this step. Organizations are highly motivated not to run their own systems. There is no competitive advantage to doing so. They want to entrust it to others so they can focus on what the business does best.
The migration will continue gradually to the cloud, with a current acceleration based on Covid-19. In a recent Gartner survey, 65% of respondents said the pandemic accelerated their plans and funding for doing digital business. Most models and strategies will be built in the cloud, and the actual runtime decisions will be distributed with some on the cloud, some on-premise in a data center, and some out in the edge in a mobile device.
Real-time streaming
In the past, traditional business information was done on static, snaps of data from the past. Today, much of this is going real-time, depending on the kind of decision that is being made. For example, when a customer is visiting a website, there are mere seconds to generate the next best offer. This is a real-time decision.
However, some decisions do not require real-time, such as the strategic decision to acquire another company or not. That is why it’s important to align the decision speed and cadence with the actual business problem.
If real-time data will be used, such as for an e-commerce situation, some of it must be streaming data. With the increase in factors taken into account when making decisions, you need data that is connected, contextual and continuous. The data must come for your entire ecosystem, not a single department, but across your company, business partners, your customers, or market data.
Streaming examples for e-commerce might include location, what the person has been doing on the web recently (clickstreams), and records of contact with your business such as calls and emails. A real-time lookup is involved with inventory and external factors like credit rating through an API, and customer data will include historical and real-time. With these factors, real-time decisions for e-commerce will be more effective, with higher yield rates and lower fraud rates.
Machine learning in model building and execution
Machine learning (ML) is making predictions, not decisions. When a prediction is made and a score is provided, a rule must be applied to determine outcomes based on the score. If considering rules or analytics, the truth is that in most cases, both are needed.
The goal of ML in decision management is to have applications that are easier to develop, faster to develop, and lead to more accurate outcomes. To achieve this objective, a process covering all stages of a decision cycle is needed —
- Observe: Getting connected, contextual, continuous intelligence. Governance is key at this step to know where data is coming from and that it will be used in authorized ways. ML models and strategies must avoid bias and alternative data sources and lending criteria should be considered to expand the business without incurring increased risk.
- Orient: The next step is putting the data in context. When dealing with models at scale, it’s important to be able to track outcomes through tools such as performance dashboards. Eventually, this will lead to the hyper-personalization of models.
- Decide: Once models are built, strategy, rule authoring, and approvals are needed. Workflow and collaboration mechanisms help manage the process and accelerate the pace of developing new decisions.
- Act: Next comes the deployment of models. Using logic to make decisions across multiple applications accelerates deployment, often referred to as centralized management or reuse of decision factors.
- Feedback: Finally, continuous logging of decisions and effects of decisions. Tracking provides the ability to audit past decisions, explain what was done, and accurately post hoc remediate. Ongoing feedback also enables continuous decision improvement at an accelerated pace.
The future of decision management includes decision intelligence
In summary, there are five considerations for the future of decision management —
- A systematic approach to decision making, including a lot more automation and decision intelligence, is clearly on its way.
- Migration to the cloud is well underway with acceleration thanks to Covid-19. Equilibrium will be reached where some decisions are made at run time at other locations, but most of the development of decision-making, modeling, and strategies will be based on cloud platforms.
- Data science ML vendors have not focused on decisions. Some may come to realize the reason you do analytics is decisions and broaden the scope of what they do, or they may stay focused and instead partner with other vendors to enable end-to-end decision making.
- For certain kinds of logic, graph databases and graph analytics can be very powerful. Likely this will become a big part of decision intelligence going forward.
- Finally, there is huge untapped potential in optimization technology to improve decisions either at development time or even at run time by applying optimization techniques. This could lead to achieving the full vision of artificial intelligence.
Related stories
