Empower the Business, but Keep Control

I was reading a recent blog from Opher Etzion at IBM in which he made reference to some of the challenges with innovation and the IT department.  He states:

Richard's question about innovation and IT department is more complicated one, however, there is some truth in his observation, based on my own experience in an IT department of a large organization, IT departments may be more conservative than their users, and typically will be cautious about be anything that is giving a "programming capabilities" to the users (and is conceived as "losing the control"). Since many event processing applications' value to the business users is doing exactly this (giving more power to the business user to do their own programming) it is sometimes a challenge to get it through the IT department, but this is a longer discussion...

While I have had many great conversations over the past couple of years with IT visionaries about CEP and event processing, I would agree with Opher that in the general sense, there is “conservatism” when it comes to exposing “programming capabilities” to their users.  There is a paradox with CEP, because the knowledge of the actual useful CEP rules for the business usually lie with the “Line of Business” or the “End User/Partner”.  While it definitely makes sense to push management and creation of the CEP rules out to those with the most domain knowledge (such as business analysts and operational business users), the thought of completely losing control of this environment sometimes “trumps” the benefits that can be made from such an agile CEP environment.  I find this very similar to the early days of SOA, when web services were just coming of age.  There was a notion that end users or business users would simply be able to consume these services at their desktop and orchestrate their own business processes.  However, the reality of “infinite loops”, “complex data mediation”, and the overall requirement to still fundamentally write “programming logic” let reality and pragmatism win the day.

At Progress Apama, we provide tools for 3 different sets of users to get involved creation and management of CEP rules:  IT, Business Analysts, and End Users.  However, the architecture recognizes the fact that as more power is exposed to business analysts and end users, there is more a chance of “loss of control”.  For this reason, we provide more of a “delegated model”, in which each layer of user (starting with IT) selectively exposes capabilities to the business analysts, who in turn selectively exposes parameters to the end user.   

For the business analyst, IT creates very simple “Smart Block Interfaces”, which capture the necessary programming logic and exposes information to the modeling environment in a straight-forward manner.  IT keeps control by exposing the data, analytics, and operations that offer the best combination of power and control for the analyst.   Apama provides a platform for IT to in effect create a custom environment for the analyst, or create a Domain Specific Language for the analyst, completely “fit for purpose” for the task at hand.  This not only allows the analyst to be extremely productive for their specific area of expertise, but allows IT to limit the risk as opposed to exposing all of the raw CEP capabilities to this class of user.

For the end user, the business analyst follows a similar process.  Based on the model, the analyst exposes parameters with constraints to the end user.  This allows the end user simply to set parameters within the given constraints to either change existing CEP rule instances, or create new instances “on the fly” with the desired parameters.  Allowing the end user to set the parameters through highly contextual dashboards provides again the right balance of power and control when getting end users involved in the process.

Finally, the back-testing environment provides the final piece of the puzzle to allow business analysts to validate their new models against historical production data.  By running the models through the historical event data, the model and the outputs can be verified prior to deploying into production.  Similarly, user based parameters can also be validated against the historical data in the same way, providing assurances that the parameters will have the anticipated effect.

In summary, I believe that the business benefits of CEP will largely be recognized “closer to the end user and the business”.  However, I don’t believe that this means needing to find clever methods of bypassing IT.  A pragmatic approach incorporates a variety of users within the organization, and makes the most out of each skill set in the enterprise.  The total result is measured by the collaboration of all the users, rather than the efforts of any specific user type within the organization.