The blueprint to optimum data integration; we sure could do with some help, right?

Crossing the lines, stepping outside the box and even lining up incorrectly are all familiar warnings; for NonStop architects, lines on whiteboards or napkins oftentimes convey complexity that needs expertise to address …

I have been involved with the building of three homes. With each home, the engagement differed as the starting place varied, but there was always a blueprint on hand. In the southern suburbs of Sydney, Lilli Pilli, the home was already standing and it was down to details. However, I did get to finish out the lower level, lay the bricks for the backyard retaining pool and come up with the design of the swimming pool. 

As for my home in Niwot, North East of Boulder, I started with a blank piece of paper and took many months before settling on the final blueprint of my dream three-level home. Every detail was covered in those blueprints when breaking ground commenced, but even so, when it came time to move in, there were additional lines drawn on the original plans for each floor that was built.

My current home in Windsor, a little to the East of Ft Collins, was yet again another instance of a home already under construction, but even so, minor details were only finalized as the house was being framed and as for the lower level, it was all my own design, although loosely based on what the builder had originally planned. Each time I built a home I gained experience and you might even say third time, lucky.

When it comes to IT, as business needs evolve, there are always plans being drawn up for the next big thing! As excited as we may be, walking up to the whiteboard to take in the scope of the project, experienced IT professionals will know that as the project takes shapes, additional lines will appear magically, almost out of nowhere, on the whiteboard. Even so, fluidity is always present whenever major projects are initiated.

I was reminded of this as I prepared an article for the June issue of NonStop Insider. Those lines, as it just so happens, are oftentimes the most complex of functionality to properly implement. It’s not just that they represent the connectivity, but more importantly today are deserving of a second look as they can mask the complexity of what is being contemplated. 

As one vendor said to me just recently, “Complexity is in the connections. Those are the lines on the whiteboard ‘architecture’ filling the void between components; between NonStop and data targets for instance, where data created on NonStop is needed in volume and on time, all the time. Interruptions will have significant trickle-on impact where the consequences of lines being broken might prove fatal for the corporation.”

As a community, those associated with NonStop have really taken to the observation by Jeff Kyle, VP Mission Critical Systems at HPE, when he said, “NonStop creates the data.” That’s right; data is created on NonStop and as such, represents the freshest and most relevant data that business will ever ingest. Fresh data representing changing market conditions is more valuable than any service provided or any commodity manufactured. Without fresh data, business will always be playing catch up and be in jeopardy of losing out to someone new to the market that is better positioned to meet the changing market conditions.

We often talk about improving business outcomes but this is not something that can occur if we are only factoring into our decision data that was created yesterday. It’s not good enough to be working with data that is that stale. NonStop systems improve business outcomes as the insight they can provide is immediate and sometimes, we don’t do ourselves any good by being quiet about this when we walk up to those whiteboards and look at the lines connecting NonStop to other systems within the business.

There are lines drawn that represent pathways to backup sites necessary for meaningful pursuit of business continuity. Disasters will continue to happen and it’s critical for business to have those additional sites primed and ready to take over in an instant. But those lines may not go to a physical data center as they may be funneling data created on NonStop to the cloud. 

Cloud service providers have made great strides in creating hierarchies of storage such that it might prove more economical to move data into the cloud than to a traditional off-site secondary or tertiary data center. However, business continuity represents only one type of line, as increasingly today there are lines connecting NonStop with data warehouses, data lakes, and more where the thickness of the line drawn on the whiteboard may indeed be quite misleading.

It’s not just a communications link but rather it may be a key component of a powerful data integration solution. And this is where the complexity can soon overwhelm less experienced IT professionals and call into question the skillsets you might be able to leverage from your favorite consultancy. There are reasons for blueprints and whiteboard diagrams and it’s all quite simple, really. There has to be defined beginnings and endings just as there are places where other interested parties have to be informed.

These could include local and regional authorities as well as internal audit, security and yes, operations management – how do you expect us to run this thing if you didn’t let us have a peak before you started the project? We could have told you that the data already exists and you are now duplicating processes – if only you had asked! When you build a house then gaining a certificate of occupancy is a major milestone. But when you develop a new application and with in-house QA behind you, it’s a successful Beta / Proof of Concept / Pilot that really informs you that all the decisions made during the life of the project were the correct decisions.

Returning to those lines on the whiteboard, one more thought came to mind as I read the Striim submission to the June issue of NonStop Insider, NonStop users rely on Striim to connect to the outside world. Azure cloud service? Yes! Kafka messaging? Yes! Data may be created on NonStop and data may need to be moved, transformed, enriched and much more, but the data created on NonStop first target destination is the NonStop’s own data base and in most cases this just happens to be NonStop SQL/MX.

As for Azure or Google or for that matter, AWS together with NonStop SQL (and the lines we draw connecting the two), such pairing is becoming the new normal for connectivity. “Clouds are capturing the enterprise IT marketplace and in doing so, they are appearing on radar screens where NonStop has already been deployed; and we already have transactions creating data on NonStop capitalizing on Striim to pipe data to Google Cloud,” said Striim cofounder, Sami Akbay.

“Databases appearing in public clouds may look attractive but what are the steps required to ensure NonStop and the cloud service providers offering you selected happily coexist without degrading the availability attributes of NonStop?” When was the last time you stepped back from the whiteboard and had a good look at that rectangle you had drawn representing a NonStop system? And where did your thoughts take you when you identified the need for a data streaming platform?

As a general rule, it is within IT where we collect data and we deliver data. The locality of data created on NonStop remaining on NonStop is very important. Not just for the integrity of the transactions that might follow or the reduced latency from not having to navigate off-platform, but for data’s own protection. Even as we shy away from attracting too much attention to the topic but data is safest when it’s on NonStop and should anything happen to data once it is passed to other systems, we can always rely on NonStop to provide that one instance of truth.

The NonStop community is well served by vendors providing products that connect disparate systems and move data – NTI, ETI, Gravic, TSI and Striim. They all look to support those lines of complexity when given the opportunity and they all do a terrific job in this regard. They all move data and they all can make getting data off of NonStop a much easier task to accomplish. All perform their tasks professionally and they all are experts in fields related to the segments of the market where they operate. 

It was from actions carried out by these vendors that the first moves to integrate data occurred and lest we forget, NonStop vendors were supporting heterogeneous mixes of data base management systems well before the rest of the industry began thinking about life with more than one data base solution.

These lines are complex and they imply so much. Next time you turn to look once again at your whiteboard don’t be surprised at just how central a role NonStop is playing in the support of that new application. No server matches the uptime or the scalability of NonStop. Surprisingly, no server can match the entry price of a new fully functioning NonStop system – have you considered that aspect of “going NonStop” of late? 

You may want to take another look as you will be surprised to find just how competitive the new NonStop X family of converged systems has become. They may be only lines and rectangles but ultimately, it’s your business being blueprinted and in a world embracing change at the rate it is occurring today, can you afford to terminate any of those lines in anything other than NonStop?