Estimations NOT based on Analysis

Today I will talk about how nowadays project managers estimate workload and create deadlines.

This Monday I got seven new requirements that I must implement in the system till Friday. Five days for everything – analysis, design, build and test. Generally it’s fine, I’m genius and professional I can do anything, but the problem here is that I haven’t even seen requirements (I’m not even talking about description), it just came right from sky, e-mail of type ‘!’  ‘seven new requirements ASAP deadline friday’ (yes, without punctuation and all lower case (I know, those things are told, like in, third grade or what!?, but this guy has finished ‘The Master of Business Administration’ (MBA)).

So, what I must do? Open the e-mail, read the requirements, analyze them, estimate… (wait WHAT? We already have deadline!!).

What I did? I went to my manager (the one who sent this e-mail) and asked, can I see data on what he based his analysis from whom he came up with this estimate, and can I see the analysis itself, as that could make my job easier. He looked at me a in a way, if I was talking Chinese to him. I don’t have any analysis, he answered. In that moment I couldn’t understand, am I stupid or… as this guy have MBA from a fancy university in United States of America while I just have diploma from post-Soviet state high school.

Okay, let’s get to the deal. I’m fourth year university drop out (yep, one of those guys), but with more than fifteen years of experience in software development. I started to learn programming (and everything about that) when I was twelve years old, back in dinosaur age, when people were afraid (and that was done by specialists) of upgrading their computers from Windows 95 to 98.

So, now you are probably asking me, okay, then what’s the correct way of estimating? Just change one word of today’s posts name. Estimation IS based on Analysis.

Simple as that.

Workflow:

1. Client says he wants new button named X on screen Y that does Z when it’s clicked.
• That’s a requirement.
2. Analysts read’s/hears requirement and does quick estimation based on his experience. This estimate always should be expanded two times (that’s also is based on experience). And this estimate should only have time needed for deeper analysis, as real estimate will be brought up only after full analysis.
• In this point, again based on experience, analyst tries to understand if client needs this thing, or is he just “thinking”, he needs. If client’s say’s he needs, doesn’t always mean he needs that, that’s the analysts job, to bring added value to client, and if this button isn’t bringing any, then there is no reason to build one (this topic is complete story on its own, that I will bring to the plate also, but some other time).
3. Management takes this estimate in to considerations and decides, will we continue or discontinue this (I like this word play here).
4. If decision is positive, analyst can start analysis. First, he tries to understand what this new button will do (functionality Z) and how that can be implemented with current functionality (on screen Y).
5. While doing analysis, he creates functional design, a high-level document that explains step by step what this button will do and how it will interact with current functionality (where it will be placed, how it will be named, what happens when it is clicked), the document that person with no programming background (client) can read and understand.
• Based on this document he creates user acceptance test cases for client with whom client will test button and approve if this is what he wanted.
6. Then, second document, technical design, a low-level document that has all the technical blabbering on how button will be implemented, in what ‘class’ code will be written, what interface if any it will trigger and what is send from and to through interface, what restrictions, rules should be implemented with button, what database fields will interact when it’s clicked, in what rows data will be saved etc. as I told all, technical blabbering. This document is the main document on what estimation can be created, as this document basically says, how much work programmer will have to do, to implement this button.
• Based on this document he creates unit/application/integration test cases what will test technical part of button, with whom programmers and testers will try to eliminate any bugs that can be accidentally created in the code by programmer.
7. When this is done – estimate can be created, how much time building and testing will take.

Problems with this approach? Yes of course, it could just take a lot of time, that’s why managers don’t like it, they better just take estimate out of the sky and base it on arbitrary assumptions. Like: in school (let’s take same, third grade) teacher gives home work for math, five exercises. You spit out quick assumption, five exercises, one exercise - thirty minutes, homework done in two hours and thirty minutes. So, now you go out with friends as you know you can spend some time with them after school as you don’t need all time after school for homework. You go back home, and start homework, then in first fifteen minutes you find out that two exercises are easy and you do them both in thirty minutes, but now, rest three are hard, and takes three hours each, yep you sit till 3am (or you can just drop them, as this is still just third grade, exams for university will not come in ten more years).

This is one of the biggest problems why nowadays there are a lot of overtime in IT projects. It’s just because estimations for work wasn’t done by the people who are doing the work. Managers only care about hitting deadlines, they don’t care about quality or proper documentation. A lot of time we don’t even create functional/technical designs, we just start programming straight in to the system without even understand, if client needs this thing. Is there added value to the requirement he is asking? If the client says he needs this new button, doesn’t mean he needs it, but that’s a story for some other chapter.

Bottom line – use your experience, technical and analytic knowledge to analyze input data and come up with estimate on how much time you will need to get output data you need – Chinese language for the MBA.

Agile

Let's start quite simple. I will try to explain why I don't like Agile. Well for first, it may sound stupid – it killed World of Warcraft. And from now on, I will use a lot of Video Game industry analogies. I, myself, have worked in quite different industries all around the world. I have built IT systems for heavy production machinery companies, telecommunications, health care, but video game industry is, how to say, one of my favorite, as being a geek.

Okay, how does the Agile killed WoW? Well it’s still one of bestselling MMORPG’s of all time’s, there’s no doubts about that, but does it still can accumulate same amount of player base as back in 2007? Sure, it’s like, what 12, years old game! But still, if it’s perfect, millions will use it also after a 100 years. Let’s say, a car! Petroleum internal combustion engine was invented back in middle of 19^th century, but it’s still widely used standard for cars. Sure, it has uncountable innovations up it since Karl Benz wife drove that first car, but the initial workflow is same, same for almost a hundred years. So, let’s go back to the WoW. Agile, tells us to release smaller things more frequently! But does it says anything about quality? Ye, maybe, I will not go through whole thing as it’s useless anyway. Quality of those small things have dropped dramatically. Back in vanilla, you had four raids, each unique and you had to found 39 more people (what was adventure on its own), dedicated as you, to beat those end bosses. Today, you have, what, a forty raids, for whom you can use Group Finder and just bounce in to raid or dungeon quickly beating it and jumping to next one, without learning about dungeons lore, and all dungeons look same and boss techniques are same. For, god, Blizzard even doesn’t care about creating new ones, they just take old ones, tweak boss health bars and present them as new HEROIC mode, basically by using same assets, but asking us new money.

Fine, let’s get back to real life. Above, was frustration number one, in a long list of my frustrations that today Doom IT.

Agile. Main point on Wikipedia says, ‘set of principles for software development under which requirements and solutions evolve through the collaborative effort of self-organizing cross-functional teams. It advocates adaptive planning, evolutionary development, early delivery, and continuous improvement, and it encourages rapid and flexible response to change.’

Let’s go through them all (we may not finish today):

• requirements and solutions evolve through the collaborative effort of self-organizing cross-functional teams

Okay, this thing, has nothing to do with Agile. It’s a normal process of everything you do in your life, been there for last four thousand years. Teams will always be self-organizing, and will cross over to other teams. What does even ‘self-organizing’ means? God damint, if you don’t know how to organize your work (any kind), you’re done, go home! People organize things they do, and human being is social animal, so he will cross paths with others always, it’s not some specific principle.

• it advocates adaptive planning

Again, basically same thing, you have to plan your work, but with catch – ‘adaptive’! It beans, if you screw up (and by using Agile you always screw up), you change your plan, and continue as nothing happened before. Of course we always have to roll with punches, but still, put this as the principle? Main principle? It already shows, that you actually don’t know how to plan your work, you just assume (ass of you and me) that you will screw up and you will have to change plan.

If you spent quite an effort in planning, and checked everything eight times, then your plan doesn’t have to fail. Sure, thing, you can create plan B (as in movies), that’s normal practice to have. But just in beginning already gave up an assumption that you will fail?? That’s wrong.

Japanese have this thing they call ‘80/20’ (I didn’t thing about annotations for the first time, but to fair and for everything to look legit, I will try to find more resources on things I will talk). Basically, it means 80% of planning, 20% of actually build. ANYTHING – software, construction, car manufacturing, doing your fitness trainings. BEFORE YOU START ANYTHING –PLAN.

You may ask, why do I think this works? I will answer you like this:

                Honda – best bikes in the world;

                Toyota – best cars in the world;

                Sony – best home entertainment electronics in the world

                Yamaha – best professional sound systems in the world

                do I have to continue…

And this all, only because Japanese are hardworking, dedicated people and they believe that planning is most important part of project, not actually building it.

Agile fails here. First, it already assumes that plan is shit, and you will have to adapt to on constant change while you’re doing build. Don’t get me wrong, change isn’t bad, change is only law of life, as one famous person told us, but, if you constantly change your plan in the middle of the build on fly, it just gets you full of stress, and in the end nothing is done, as final result is nothing that was initial planned, but are compiled from different parts, each part from the each plan, you changed. Yes, take a glue, stick it all together and let’s hope it doesn’t break apart, LOL (that’s another story, on how companies nowadays implement IT system support workflow (by assuming that system will break in the first day of the release), will take note on that).

So, now some thing from my personal experience. Oh gosh, how much I haven’t bounced back and forward, just because, project hasn’t had a real plan… Manager creates plan (let’s say in Gantt chart), gives each team member tasks, but adds, when you start building, you can easily add a tasks to your plan… WTF? Did you then created a plan or no? Or do I have to do it on my own? If so, then mr. Manager, I’m done! Yes, I finished my tasks, open your plan and check! No, no, no, I don’t have to do those task, they are not applicable, I changed the plan, and you told us we can do so!

Then, next thing happens. By Agile we split build in phases (yes, we don’t split project in phases, we split build in phases, whatever that means). Phase one, you do plan, small analysis, you do build, unit test, done. Not done? No problem, we will just roll over thins thing to next phase! End of the last phase comes, AND WHAT A SURPRISE, you have about 75% of not done tasks from previous phases. Why did it happen? Because we don’t validate end of the phase, and start next phase before we finish first one. Imagine car, you have to change brakes on it, but you put yourself on a deadline, 15 minutes per task, so you took off tire, it took you 2 minutes, so now, you smoke for 13 minutes. Then you change brakes, 15 minutes passes, but you haven’t yet done half of it, aaa screw it, let’s roll to next phase, PUTTING TIRE BACK ON.

So, now team is full of stress, as they have to build all previous phases in one phase and the deadline is close. Yup, we use ADAPTIVE PLANNING.

• evolutionary development

This is just marketing use of words, that doesn’t have any meaning besides this: Client asks, how will you build my system, manager answers, we will use “evolutionary development”, client blushes, as he has no clue what that is, but understand that “those guys” are professionals.

Okay, I can’t take this anymore, let’s continue next week.

Lessons Learned

Before I even start, I wan't to get some things straight.
1) I don't like agile. https://en.wikipedia.org/wiki/Agile_software_development
it's the one of the reason, why nowadays software comes out full with bugs, and a lot of time even don't come out at all, but got postponed.
2) It's been almost seven years since I can call myself, a professional programmer. Before that, as amateur, I can go back to my tenth birthday, when I got my first computer with Norton Commander on it.
3) I'm total nerd and geek, by owning more than two video game consoles and two high end gaming PC's

Now, when we get out of the way things that you need to know about me, I will start. As I already mentioned, I have been working as a professional programmer (or software engineer, what ever suites you best) for last seven years (after I partly finished university (partly - that's another story)). And the biggest problem is that none, I repeat, none of the projects I have been working on, have not been postponed before their initial go-live date. NONE.
So, I just thought, maybe, I need to gather all this bad things I have encountered in those past seven years, and share my frustration with other people (if others do care as much as I do about this).
In this blog you will see a lot of me trashing on Agile Software Development and IT Managers. It's not that all of my managers where bad, not at all, some of theme where great persons and interesting friends, with whom I have built great relationships, but they are not good IT, I will repeat, IT, managers.
I'm engineer, and as engineer, I'm really picky and punctual, and when manager says to me, that project don't care how we name classes or tables, because, project doesn't have any naming convention, that drives me nuts. And when project implements new change control workflow, and also DRAWS a big MS Visio document based on that, but after two days, none of the people in project follows this, as it includes getting a lot of approvals, to implement your change and put it in the production. Manager just says, 'document is just guidelines, we don't have to follow it specifically'. Grrrr......
I will try to do post a week, and will not try do them really big, like, no more than 15 minutes of writing, and for you no more than 3 minutes of reading. Hope it will be interesting for me, and for whoever will be reading this, if anyone :) !