Sunday, March 6, 2011

Too much clutter!

This is not the usual ramblings on my part focused solely on issues to do with NonStop. Nor is it a pipedream on my part! All I am suggesting in this post is we give due consideration to throwing away much of what we run today …

When plans were finalized to return to Boulder and to work from the home office, it didn’t immediately occur to me that this would involve moving a lot of stuff around. So much so, that after a couple of days I gave up and called a trash collection contracting service whose truck I had passed only a few days earlier.

All the same, it was a heart-wrenching moment when it came time to make the call as I was going to be discarding not just empty boxes and packing paper, but a couple of pieces of furniture as well as a several appliances I just hadn’t used for the past ten years. Rather than just adding more stuff to a growing collection of what at one time I had valued, the time had come to take a deep breath and just throw out the lot.

Dan and Tim from 1.800.GOT.JUNK pulled up in my driveway and after I directed them to the piles of discarded former treasures I had assembled at the end of my driveway as well as outside my basement, they quickly set about tossing it all into their truck. The picture above is of the process under way with Dan and Tim carrying stuff from one of the stacks! It felt good knowing that at their regional location my “stuff” will be sorted, with items of value likely to find new homes.

Knowing I am working on a blog about clutter my wife, Margo, after visiting a hair dresser’s salon, brought home with her the March 2011 issue of the magazine. Oprah had written in her editorial page, “What I know for Sure” how her life was “filled with so much clutter.” As she moved on to a new project, she admitted how she was “not just cleaning my closet. I’m cleaning out my life.”

As she described how she was going about this she remarked “I’m keeping only that which delights me or embraces my well-being. That means another big ol’ garage sale coming up: clothes, shoes, dishes, furniture, stuff.”

When it comes to our work, particularly when it has to do with technology and running the complex data centers, there’s a reason why we use the expression “let’s start with a clean sheet of paper!” And it’s no wonder we all have empathy for the team that truly can “think outside the box” – as for the rest of us inside the box, there’s just too much stuff to see clearly! There’s too much clutter we have accumulated that often can get in the way of making good decisions!

It’s also an admission that what we have, and what we continue to do has become a distraction, and before we can put together any plans for new projects, we need to clean house!

While editing a posting to another blog, there were questions as to why I had introduced two storylines and over a couple of paragraphs tried to tie them together, when all that I had succeeded in doing was developing a rather confusing, and somewhat cluttered, storyline.

There’s still no escaping the value of keeping something simple and I was left wondering, whether it’s undertaking the simple task of writing or, equally, overseeing a complex data center, as Oprah had written, “enough already with the stuff!”

When we take a good look at the mix of software we run today, is it holding us back from pursuing routine upgrades? How often have we heard that a particular routine or program has to continue to be run as it’s always been a part of the solution only to have no one able to explain what it does or who even developed it! And yet, it’s too risky to pull the plug on it, as managers are very nervous should something break – a lot easier to just let it run than do forensic to uncover its history!

Of course finding the time to pursue deep investigations on all the code we support is just something we do not have time to pursue – after all, we are all contending with the mantra to do more with less and superficially, this doesn’t look like a priority. Are we expected to rewrite the whole solution? Wouldn’t it be a lot simpler to just keep running until we replace with a packaged solution?

For the most part, this is what does happen inside many data centers but the stuff we have accumulated is beginning to weigh us down. We can no longer find engineers who could even analyze what it all means should something go wrong and besides, with the little staff we have today, we are over-committed just to maintain all this stuff – we can’t touch much of it, as it’s just too important to business operations!

Today, the focus has shifted to modernization and to convergence – after all, how else we will capitalize on the benefits on offer from vendors such as HP if we can’t align with their product roadmaps . As technology leads us deeper into commodity platforms and solutions providers deliver products built with popular frameworks supported by the latest programming language offering, where will we find the time to consider and evaluate, let alone deploy. We have all this other stuff we just have to look after!

Isn’t it time we planned on throwing the whole lot out? Isn’t it time we called 1.800.GOT.CODE? But is this reasonable or even possible? I asked a recent start-up infrastructure vendor how quickly they produced code. How feasible is it to simply replace a product or solution? As this vendor has a small team of engineers with “history” at two other companies pursuing similar products, it was very important to them that everything be developed from scratch.

Inside of twelve months they had written a new product with more than 100,000 lines of code, sufficient to where Proof of Concepts (PoC) could be pursued. As this vendor moved through the PoC and completed supporting additional capabilities, as well as responding to customer requests for new features, the product grew to 250,000 lines of code. A completely new, alternative product offering written using all the latest development tools, frameworks and libraries and as modern as anything else on the market – all completed in just over a year!

Dan and Tim filled their truck and drove away but only after agreeing to come back the next day to collect even more junk. There’s something uplifting about seeing all the stuff you no longer value being taken away. Cleaning house has let me focus on what’s important to me even as it frees up so much time to get on with other things I really like to do!

It’s time we really clean out the old and give serious consideration to replacing it. With hindsight, how much of a contribution to our companies are we really making when we spend seventy percent of our time maintaining stuff we aren’t all that sure is actually doing much of anything!

Unrealistic perhaps and obviously I just don’t understand the implication of what I am saying and yet, on the other hand, how many IT departments have really tried? The tools and languages on offer today will, in most situations, allow us to unload so much stuff we may be very surprised. We will begin to see how much of a distraction it had become and how it’s very bulk had prevented us from having a clear view of where we need to be.

Imagine how much time will free up for development projects we really would like to do. And if we can just pull back on what we maintain to a lowly fifty percent, just think of the bigger contributions to the business we will be making!


Lyman Lundquist said...

Richard, as you are keenly aware, this most recent topic strikes very close to me. For the better part of nearly 2 decades, I have been involved and faced with exactly as you describe and that said, I wanted to add some comment. First off the story of the "product" that started at 100Kloc (thousand lines of code) and ended at 250Kloc actually took 2 years, not one. (Initial development and PoC was, according to your detail, completed at the end of year 1). What is missing here is the cost for the 2 years of work and the differentiation between "new" versus "replacement" function. Throwing out the old in favor of new must focus on obtaining the EXACT same results and this is the single most feared issue a customer/user/corporation faces when considering transformation, modernization, etc. You are 100% spot on in the concept / desire to modernize because of the rapidly dwindling skill / resource pool / subject matter expert upon which the current user must depend. But as each environment (h/w and s/w) over the past several decades is at some point uniquely different (big endian vs little endian, ASCII vs EBCIDIC, open system/source vs proprietary, green screen vs GUI, BAL vs TAL, relational vs hierarchical, legacy language vs JAVA, Ruby, Python, etc), getting to ones comfort zone / assurance that all will be right with the world, once the old is out and the new is in...IS the biggest hurtle for everyone both current users and those that tout the ability to "seamlessly" solve the above, and other, design/implementation issues. This situation normally maps ultimately to "cost" and even with current "technology" (tools), a 100% assurance guarantee is not available. Now, in your comparison, you dumped stuff you had either "duplicates" of or have not used / dont plan to use. FARE! much "sunseted" applications exist today that should be pruned (thrown out)? But if you were faced with obtaining a "100% functional replacement" for the appliance(s) that you discarded (lets say your life depended on the use of same), then you might even consider holding onto them, even if one was a backup..or at the very least until you tried out another "advertised functionally equivalent, newer, better, easier to use, faster, more modern, etc." appliance before you cast off the old. Most environments of any type of legacy today are in the same spot..between a rock and a hard place, as the time, cost, risk and the impact to the current/future business, to cast out the old and replace with the new, has not been suitably resolved (business case). Solve this 3 variable equation in a proven, repeatable, definable manner and you have a winner on your hands!

Just a view from the cheap seats...

Justin said...

Well good on ya for attacking the clutter. I believe there is a point about old code that needs to be looked at. I'm wondering if some of the Malware test code or even some of NonStop Measure info could at least find code that wasn't ever being executed. However what surprised me the most after reading you for years is that within the NonStop blog we'd have Oprah quoted. Now that's outside the box!

Harry said...

Richard & Lyman,
In the NonStop space there are “proven, repeatable, definable” solutions to achieve application de-cluttering and it all starts with modernizing the database.
According to statistics we continue to hear, as much as 70% of the databases in production use on NonStop are still Enscribe. The net result of this is that organizations have to design, develop and maintain virtually everything themselves because Enscribe is proprietary and there are no industry standard tools/technologies that support it (nor will there ever be). An example is Eclipse – if you have Enscribe you cannot use it.
If you could magically replace the Enscribe files with NS SQL tables, then you could immediately start using off-the-shelf tools, even open-source, to functionally replace piece-by-piece the old with parts that you don’t have to pay to design, build & maintain…Each step, each piece, making the whole better and less costly.
Quite a number of customers have actually done this type of incremental modernization, some using our Escort SQL product, some have done it manually. As you’ve said, the key is finding the tools for your environment that make the most of your time, lower your costs, and help you achieve this modernization/de-cluttering in a way to avoid project risk while every step of the way improving your current/future business.

Kevin Christian said...

I met with a NonStop customer recently who must decide how to innovate without risk to operations. Cutting clutter is important. But using fresh eyes to ask on a scheduled, recurring basis "What are my mission-critical elements, NOW?" should highlight chunks as candidates for instant discard. Go for the big stuff first. Then you'll find time to both innovate and declutter the mission-critical core.