Skip to main content

I need data I can digest, in small bites, please!

When it comes to the NonStop community discussing the potential benefits from Big Data often results in conversations about taking baby steps and where less big may be the place to start! 

How many times have we heard the expression “biting off more than we can chew”? Whether it applies to household chores, car maintenance, or simply arranging our next vacation, there’s always something that comes up that belays expected benefits and rewards. With summer and the time for BBQs, this expression often comes to mind.

There’s no denying that IT is littered with failed projects and the more I talk with IT professionals, the more I sense that we continue to bite off more than we can chew. Even among the NonStop community, it’s not uncommon to hear of projects, from simple modernization efforts to major system upgrades, coming up short and often simply abandoned.

This week, a couple of emails arrived that had me thinking about past failures and they were all about Big Data – multiple invitations to read papers, participate in surveys and join webcasts. While it’s true that Big Data is among the most talked about topics across IT, it’s also a little overwhelming and the opinions from experts do little to lessen our anxieties. For a while it seemed that the NonStop community could pull its head in on Big Data, but now companies are wanting to elicit marketing “truths” as they develop in real time and NonStop is in the cross-hairs of every business manager targeting exploitation of Big Data in real time.

First up, I received an invite from HP to join HP CEO, Meg Whitman, and her team working on Big Data “as they discuss the pressure to extract valuable insights from your data even as the volume and variety of data being collected significantly increase.” There’s those two critical V’s again that along with variety (and occasionally veracity), help define Big Data. However, this wasn’t the item that caught my attention. It was the reference to there being “the pressure to extract valuable insights” within IT as it continues to store all the data it captures. So much in fact that apart from those selling storage, nobody seems all that sure if there’s anything good going on at all!

Secondly, there came an invitation from IBM to “tell us what it takes to create a data-driven competitive advantage”. When you dig a little deeper into the invite, IBM let’s on that it is “seeking a cross-industry, global pool of respondents who have business or technical responsibilities for analytics activities.” What followed was a survey IBM created from which it hopes to be able to get a better sense of what business really does need.

The third email was a link to a HP post,
No limits: How Big Data changes competition - Data drives the bottom line, and technology is no longer limiting your competitors. “Data-focused businesses have rejected the concept of compromising business objectives for the sake of technology,” said HP. HP said it after noting that “For organizations that weren’t ‘born’ in the era of no-limits technology, a transformation is required.” Furthermore, HP adds, “It’s a novel idea for most organizations, but it’s in the DNA of young, agile companies.” And, “To compete, the rest of the market will need to act urgently to change their data ideologies and reject limitations as they store and explore data, and serve analytics insights to the business.”

The one thing I believe we can all agree on is that everyone has an opinion about Big Data, but here’s the problem as I see it; is this simply a case of there being more Big Data than we can chew and if so, is what we would really like actually less Big Data? Very few enterprises today feel comfortable with committing to three or five year projects and are more interested in immediate gratification – so, can we reduce Big Data into bite size chunks and indeed, can we tighten the focus data-driven apps and apply to lesser, more mundane tasks?

But again, here’s the rub and I have written about it in other posts – Big Data is not an off-the-shelf, one-size-fits-all solution. In fact, it’s not really a solution but rather a tool that, in the hands of knowledgeable business scientists, can perform amazing feats and deliver incredible business insight. For most enterprises I know, such business scientists are a non-entity. Where they do exist they are a part of a vendor and are focused on making sure product A outperforms product B. And, as an aside, I am pretty sure that, over the long haul the model of capturing then storing then analyzing data is not a sustainable model – capture, analyze, store seems to be a little more workable for the enterprise.

Among the NonStop vendor community there’s already inklings of what lies ahead – solutions and middleware vendors are beginning to add capabilities that tap Big Data. IR with Prognosis has already confirmed that it’s at vendors like IR “where you will find the data architects, statisticians and data scientists that can decipher what’s taking place from the myriad of events coming from multiple sources.” In the May 6, 2014, post to realtime.ir
For business managers today, situational awareness is critical … IR’s Jay Horton then added how, “The Business Insight provided by IR, is as big as the customers want it to be, and is limited only by their perspective on what is important to measure.” Alerts produced before a crises starts is always more preferable than an alert that simply states the obvious – you have been robbed!

As for the solution vendors, it’s hard to miss the messages coming from OmniPayments CEO, Yash Kapadia. According to Yash, in the March 12, 2013, post to ATMmarketplace
You don’t have to make it to 6th grade to know EMV support is the smart thing to do! ,when the discussion with any prospect turns to fraud, then “my experience tells me that work being done in adjacent technologies such as analytics and the accumulation of data in Big Data frameworks is going to be leveraged by payments platforms in new and innovative ways. Legitimate customer experience cannot be compromised, of course, but the need to detect potential fraud as it is being perpetrated has become of paramount importance to all in the financial industry.”

Expressing similar sentiment to those by IR, Yash also notes that across his constituency there’s “little expectation that customers will have the ‘Big Data’-skilled scientists at hand and are looking to OmniPayments to capitalize on Big Data (where it’s in place and accessible) and furthermore while I am not expecting to see Big Data implementations on NonStop systems I am expecting to be a consumer of Big Data analytics generated in real time.”

“No transformation required” is well understood at WebAction. “To be completely honest, when it comes to enterprises relying on NonStop systems for mission critical transaction processing,” said WebAction Executive VP, Sami Akbay, “we know of no instances where embracing the value of Big Data has seen such an enterprise replace their NonStop system. Quite the opposite, in fact, augmenting a transaction and giving it access to additional information for greater insight seems to be a reasonable request and one we are taking seriously.”

While WebAction isn’t changing the underlying technology of its core product, the messages in support of WebAction have become more tightly focused of late – visitors to the
WebAction web site can now view a number of examples that are of interest to any enterprise running NonStop systems. “Whether it’s just a case of optimizing data center management along the lines IR has recognized or enhancing security event processing where OmniPayments sees concerns, we are seeing enterprises building, modifying and then deploying real-time apps in days and not months or years.”

Should you be interested in following the discussion about Big Data and its intersection with NonStop and real time transaction processing, you may want to join the new LinkedIn group, Big Data, integrated with NonStop. While there is no Connect SIG supporting Big Data at this time, this may do as a substitute.

Biting off more than you can chew has plagued IT for as long as IT has existed. The five-year Mega projects that make headlines have a poor history of completing on time and meeting expected requirements – not hard to imagine as IT just hasn’t ever stood still for five years. Vendors selecting elements of Big Data they see as helpful and enhancing the capabilities of their products also makes sense and is something easily comprehended. New vendors building tools to enhance the experience of consumers interacting with mission critical real time applications is also easy for most of us to swallow.

Less big might be right up there with giant shrimp and user friendly but to those who have witnessed change within IT over decades, breaking down a new technology into bite-size chunks makes sense. Big Data is above all else, big! And as such, demands transformation even as it favors those companies starting from scratch. However, when it comes to the NonStop community and the enterprises they serve, the risks associated with ripping and replacing isn’t attractive and seeing Big Data arrive incrementally is a godsend!

Comments

Scott Healy said…
This blog highlights an opportunity I faced a number of years ago while running extremely high volumes of airline shopping queries through NonStop being monitored by IR. We were literally dumping valuable log data in the bit bucket. Locked within the log data was what the traveling public was asking for, what they were shown, and what they bought with regards to air travel. This stream of data, well actually more like a raging torrent of data, could have been analyzed and used to create actionable, real time business intelligence for airlines with an application built on Webaction’s real-time enterprise data app platform. A lot has changed in the last decade – it will be interesting to see which companies can exploit the capabilities afforded by new, and perhaps some reborn technologies.
Dean Malone said…
I see the need for a few critical pieces to make this possible. The watchword here is "building blocks." It starts with the plumbing. Middleware is needed to move data from its source to its destination. In the case of NonStop, that is shaping up to be seen as Nonstop-to-HAVEn platforms.
Dean E Malone said…
That middleware had better be fast because hey baby, it's an InfiniBand world that is emerging. That middleware had also be asynchronous because blocking to wait for something (i.e. a queue, resources, write completion etc.) is not an option. Speaking of resources, these resources are typically shared so a mechanism is needed for synchronization (i.e. serialized access, group notifications, publish-subscribe etc.) The final requirement is that this middleware needs to be usable by applications. I am envisioning an OLTP application that generates an actionable event that need not be part of a transaction but does need to be dispatched in a very light-weight manner. SQL-based pub-sub is not going to cut it.

Justin and I were talking last week about the notion that Big Data needs to intersect with Fast Data.

Finally, look at what is in place right now. We have Moonshot that supports Mellanox InfiniBand cards and integrating it with Windows and Linux is already a done deal with open-source VMA. Nonstop, VMS and HP-UX need to be able to play in that space because though they may not be presently available on Moonshot, there is no reason why they cannot have a fiber connection into a Moonshot platform. This means its access to Moonshot resources will be just as fast as that of all the processors residing in the Moonshot cabinet. What is missing is higher-level middleware solutions that can leverage this plumbing. If HP builds this, they will come. Other thoughts?

Popular posts from this blog

If it’s June then it’s time for HPE Discover 2021.

  For the NonStop community there has always been an annual event that proved hard to resist; with changing times these events are virtual – but can we anticipate change down the road? Just recently Margo and I chose to return home via US Highway 129. It may not ring any bells, but for those who prefer to call it the Tail of the Dragon – 318 curves in 11 miles – it represents the epitome of mountain excitement. For Margo and me, having now driven the tail in both directions, driving hard through all these turns never gets old. Business took us to Florida for an extended week of meetings that were mostly conversations. Not everything went to plan and we didn’t get to see some folks, but just to have an opportunity to hit the road and meet in person certainly made the 4,500 miles excursion worthwhile. The mere fact that we made touring in a roadster work for us and we were comfortable in doing so, well, that was a real trick with a car better suited to day trips. This is all just a p

Three more wishes coming soon – the path ahead for NonStop.

So, another three years have passed by and I find myself writing a preview of what I will likely focus on in eighteen months’ time – my next three wishes for NonStop! It wouldn’t be fair on my family if I said 2019 had been a routine year for Pyalla Technologies. It started with the return flight from Sydney, Australia, and continued with three separate trips to Europe plus a lengthy road trip to Las Vegas for HPE Discover 2019 combined with stops in southern California and participation in N2TUG back in Texas. The miles have added up but all the while even as the adventurous life continued to unfold, there was so much news coming out of HPE that scarcely a day passed without a discussion or two over what it all means. Margo and I have our roots firmly anchored in NonStop, dating back to Tandem Computers where Margo had risen through the development organization all the way to the COO role under the stewardship of Bill Heil when Bill headed the NonStop Software BU. As for me

ACI Strategy - it's all about choice!

I have just returned from spending a few days in Omaha attending the annual ACE Focus meeting. These two day meetings provide more in-depth technical coverage than is usually found at the regular ACI user events, and ACI customers have been coming for more than a decade to hear the messages directly from company executives. The picture I have included here is of the venue of the Wednesday night social event – a reception held at a local sports bar called the ICEHOUSE. And I found this extremely ironic as my own involvement with ACI came through my association with the ICE product. For most of the ‘90s, ACI had been the global distributor for ICE and then, as we began the new millennium, ACI purchased Insession, creating a separate business unit that it named Insession Technologies. For nearly six years, as part of ACI it enjoyed a successful partnership with the NonStop community and had provided a number of solutions in communications, web services, and security. But the decision in l