The Hidden Power of Lookup Tables in Decision Automation

Introduction

Decision Tables in Decision Automation

The decision model and notation (DMN) is a powerful means of automating complex, operational decisions while keeping the logic of these decisions transparent to business users, accountable to stakeholders and quick to adjust in a changing marketplace. Using DMN, corporations can separate the business logic that defines their customer’s experience from the impenetrable infrastructure code, of their operational systems.

As a result of this separation, business logic can then be managed with decision aware tools, which means all stakeholders can understand the decisions. As a result, they can monitor and change their business performance as required. Furthermore, proposed changes can be tested and assessed for impact before deployment, giving greater assurance.

Importance of Decision Tables

Decision tables are often used as part of the definition of business decisions because they are concise, easy to understand and easy to change. Decision tables are also highly accountable and traceable: it’s easy to see why a specific outcome occurred. For these reasons, decision tables have been used for decades as a trusted means of representing simple decisions or contributing to more complex ones.

Each decision table has a set of inputs. It uses the values of these inputs to determine its output (of which there is usually only one).


Figure 1 Decision Table to Determine Maximum Student Loan

Decision tables express a set of conditions on the inputs’ values. The set of conditions that is satisfied determines the output in a given case. For example, consider the simple decision table in Figure 1, which determines the maximum tuition loan (the output) that can be made to a student depending on the academic year, the type of the learning institution and whether the study is part-time or not (the inputs). Note that each column represents an input (blue), apart from the last, which is the output (pink). Each row represents a set of conditions. Whichever row is satisfied by a student’s details determines the value of the output, the maximum loan value in this case.

Looking at this example, it is clear to see the power of decision tables. Their representation of business logic is compact and easy to understand.

Large ‘Lookup’ Decision Tables

Occasionally, the logic of business decisions requires decision tables with many hundreds or even thousands of rows. It’s easy to imagine how this might occur in our example. If we added the student’s age as an input and covered ten academic years, the table above would span 800 rows or more.

One way to overcome this is to use a crosstab style decision table. Rather than the one-dimensional search shown in Figure 1, this uses a two-dimensional approach, see Figure 2. A DMN relation can also be used to achieve a similar effect.


Figure 2 Crosstab Decision Table

This arrangement trades off columns for rows and allows for a more compact representation of decision tables. However, 800 separate tuition limits might still be challenging to enter into such a table.

Typically, the logic of larger tables originates from databases and spreadsheets covering matters such as pricing discounts, risk adjustments and eligibility tables. Because these tables look up a single result given many input conditions, they are called lookup tables.

These simple tables can get large if the number of combinations of input values is large and there is no systematic way of calculating their result. Voluminous lookup tables are an essential part of pragmatic decision making but are impractical for manual entry into a decision model. So how can we manage such large tables?

Managing External Lookup Tables in Genius

RapidGen recently released the first version of their efficient decision automation suite, Genius, to support DMN 1.3. The Genius product can handle large lookup tables without requiring them to be entered into a decision model. Instead, these external tables are referenced in the model and executed by Genius when needed. This means:

  • Large, existing lookup tables can be integrated into decision models with no need for re-entry or even cut and paste – even the most extensive tables can be integrated in minutes.
  • The logic of lookup tables is subject to the same high standard of verification as all DMN models.
  • The lookup tables can reference and execute existing spreadsheets, CSV files and other external data sources, allowing for rapid turnaround of logic changes. The table logic is maintained using the appropriate toolset (e.g., Excel), which supports large tables more effectively than decision modelling tools. Therefore, business stakeholders can see (and amend) their logic in the tool most familiar to them
  • One-dimensional and, soon, two-dimensional lookups can be achieved (see Figure 1 vs Figure 2).
  • The decision model can manage multiple lookup tables in one central location.

Lookup tables are very efficient. For example, Genius can execute lookup tables with 2000 rows more than 300000 times per second[1]!

 

Doesn’t That Mean My Logic is in Two Places?

With externally sourced lookup tables, there is the concern that the decision logic is now in many places. In other words, business logic is distributed between the decision model and the spreadsheets, CSV files and database it imports.

This is rarely a problem in practice because the integral relationship between external lookup tables and internal decision elements is maintained in the decision model. External sources can be represented by hyperlinks and reached in one click from within the model. In any event, the detailed content of very long lookup tables is seldom directly inspected by decision modellers. Instead, they rely on unit testing and traceability of decision execution to maintain integrity. Genius supports both.

How Do We Update Tables in a Disciplined Way?

If the source of the lookup table changes without warning, doesn’t that mean the decision will unexpectedly start to behave differently?

Only if that’s the behaviour you specified. Genius distinguishes two styles of lookup table import: pull and push.

  • Push import means that the decision always loads the latest contents of external tables before execution. This means it’s always up-to-date with the newest version of the imported logic. This allows logic to be updated without any need to revisit decision models. This approach is ideal for agile development.
  • Pull import only imports an external table when you explicitly request it. Even if the external source changes, Genius will continue to use the last imported version. This is ideal for disciplined, deployed systems that require managed change.

How Do We Check External Logic is Consistent?

If the logic is acquired from an external source, how can we be sure it is consistent?

RapidGen Genius is DMN 1.3 compliant and comprehensively checks the logic of your table at import time. Any logical inconsistencies are immediately reported.

Conclusion

Decision tables are a transparent (simple to understand), highly accountable and trusted means of representing business logic. However, decision tables can get large in some applications. Lookup tables provide a powerful and pragmatic new facility to support the rapid inclusion of decision table logic from external sources into decision models without re-keying them. They allow business users to safely incorporate external logic sources into their decision models with minimal fuss. Furthermore, they permit large lookup tables to be maintained by stakeholders using the tools with which they are already familiar. Finally, RapidGen Genius supports high-speed execution of these tables even with modest hardware.

[1] i5-4570 3.2Ghz processor with 16Gb memory running Windows 10 Pro.

About the Author

Jan Purchase has been working in investment banking for 20 years during which he has worked with nine of the world’s top 40 banks by market capitalization. In the last 13 years he has focused exclusively on helping clients with automated Business Decisions, Decision Modelling (in DMN) and Machine Learning. Dr Purchase specializes in delivering, training and mentoring all of these concepts to financial organizations and improving the integration of predictive analytics and machine learning within compliance-based operational decisions.

Dr Purchase has published a book Real World Decision Modelling with DMN, with James Taylor, which covers their experiences of using decision management and analytics in finance. He also runs a Decision Management Blog www.luxmagi.com/blog, contributes regularly to industry conferences and is currently working on ways to improve the explainability of predictive analytics, machine learning and artificial intelligence using decision modelling.

purchase@luxmagi.com   |   @JanPurchase   |   https://www.luxmagi.com