Wednesday, February 5, 2014

A surprising effect of biggish data

We had a fairly important go live in the last couple of weeks. Data entry into a purchased front-end system with near real time feeds to a back end data store. Pretty standard stuff in many ways. But one of the big ahas came when we realized that by having all fo the data from the external system (every state change of the important entities), it meant we could quickly research issues without having to go back to the system of record.
We could run balances, filter test transactions that might have been in the system from the final tracker tests, show key metrics, you name it.
The exciting moment for me was when a business user asked if we could identify transactions that may have been in error - the front end was saying something "funky". I heard the request and asked, "When would you like to know?". Business response (people not used to having truly up to date data) was, "Do you think you can get it to us tomorrow evening?" My answer, "Well I could delay it that long, how does a few minutes look to you?"
And sure enough we were able to deliver answers that quickly.
So, for me and my experience, the ability to perform real time analysis of the data, to discover glitches early and quickly, without the need to recreate made the solution worth its weight in gold. An unexpected bonus.
Of course that same data set is used for creating KPI and other operational reports too - the troubleshooting is an added bonus.

No comments:

Post a Comment