Friday, November 28, 2014

Big Data and Operational Rules

As usual, a story to start things off. Years ago I used to commute a couple of times/week between Dallas and Houston. SOP was to book home to Dallas on the last flight of the evening. That way I would be sure to make it home. However, on several occasions I would be done early and want to get home early. The less-experienced traveler would look to stand by on an earlier flight. Join the throng of people looking to move about on a crowded route.

The more experienced traveler would examine the refund policy. Realizing that the late ticket was refundable, the traveler would purchase a ticket for an earlier flight and do the refund later. Yes the purchased ticket might be a bit more expensive, but why not? That's what expense policy is for.

The very experienced traveler would look for the most overbooked flight earlier than the originally booked flight. Assuming overbooking, the traveler has the chance of offering to be "bought out" and get some form of compensation for being inconvenienced. It had the added bonus that one would be moved ahead of the stand by passengers for the next flight as well. So, the simple act of making tickets fully refundable can enable some quite naughty behaviors.

Now to the point:
Should the airline recognize this kind of behavior and do something to stop it? Sadly the answer is, as usual, it depends. What does it depend on? The balance between good will (especially in this time of over-sharing via social media) and revenue loss.

It's easy to envisage the good will part - handy too because nothing has to change, so no thinking required. However, if there is a real problem, then something more must be done. The environment must properly enable the activities to be performed.

One approach is illustrated in the diagram below, there is a transactional system whose behavior we wish to influence. This system (and several others) send their transactions to the Historical/Analytic Data Store (HADS) which does very little formatting. It is after all the full record of the transactional history of the enterprise (Now that is BIG DATA!). The transactions are also sent to the Operational Analysis System in near real time. The Operational Analysis System is backed by an Operational Data Store (ODS) which contains the most recent state of the operational transactions. The Operational Analysis System's main role is to perform analysis on the transactions in its domain and, if necessary, take action against its transactional system. It uses rules to drive the analysis.



Simplified view of Analytic and Operational Elements
I
So, back to our example: How do we know how widespread the practice is, and what it has cost the airline? Where do we go to dig this out? Use the Data Analytic System fronting the HADS. We should be looking in the HADS for the residue of inappropriate behavior over time. If we find some, our next step might be to recognize the elements that make up the behavior, deducing the necessary rules, and placing those rules into the Operational Analysis System. At the most extreme, we might see the Operational Analysis System issuing cancellation transactions against the Transactional System. Of course these cancellations will then flow just like any other transactions.

Bottom line. We need to have the ability to detect patterns over time in our transactions; determine the value/cost balance of these patterns and, if required, implement rules that can detect and act on these transactions on the fly. No need to "crack the case" on the Transactional System - that system might be a purchased/outsourced system or it might be a system of record. We need to make sure that it is capable of receiving and executing transactions generated by the Operational Analysis System, so it continues to be a proper system of record.