Tagged with: agile, Bookmarks, energy
Tagged with: agile, Bookmarks, enterprise, Linux
As anyone who reads this blog regularly knows, I’ve spent a lot of time over the last 3-4 years studying agile methodologies and most recently lean concepts and principles. I have most recently been reading a couple of books by Ricardo Semler, who runs his company in a completely democratic way - doing away with all top down authoritarian management principles and allowing the employees to make decisions on dress, salaries, where they work, when they work, and most importantly, how they work.
I remember when I had first read the book Agile Software Development with SCRUM by Ken Schwaber and Mike Beedle and I had sent an email to my manager with a link to the book and the small sentence fragment “common sense codified”. We began experimenting with Scrum within my group, but it was very difficult to get other groups on the same page. We wound up with a lot of sprints that ended when things left development and entered the “normal corporate process” to finish things up and get them to production. We also had a lot of conversations around whether what we were doing was “standard process”.
As I started reading books on TPS and Lean, the same thing occurred. It struck me how most of the things that are characterized by “lean” are just common sense principles explained in such a way that they sound like a “process” that manager types can “buy into”. But really, they work because they make sense - and people have the permission to standardize and then change their work rather than having things written down and subsequently treating these processes like they are set in stone. You can’t change them unless you go through an agonizing approval process up the management chain.
Over the years, I’ve read and listened to many podcasts talking about the same things going on in other companies. People struggle to be able to change the way they work because they have to get “management buy-in” to take action. So much wasted effort just to try something new.
Interestingly, once the “buy-in” occurs, over and over again people try to “implement Scrum” or “do XP”, mostly by the book, and do not throw out things that do not work for the group. For some reason, we all think we have to be a part of some “methodology” in order to be effective.
One of the most fascinating things about Semlers company is the explicit trust and ability to control ones own destiny that the employees of Semco seem to receive. I ran across a particular section of the book where he was talking about the process improvements that “just started happening” due to this culture change and I found some interesting similarities to lean that I thought would be interesting to highlight. He starts off with:
The factory committee spun off groups that studied the plants products and how the workers made them, looking for ways to save time and make improvements. These teams weren’t created by Semco; they formed spontaneously, as the bracing winds of democracy swept through the food service equipment unit, and often met after hours or during lunch.
Interesting, employees actually wanted to do a better job and self organized because they could.
He goes on, talking about some of the changes:
One group restructured the dishwasher assembly line, changing it from a sequential assembly process to a batch concept in which dishwashers are assembled in twos and threes by teams of workers that do many different tasks and spend the time between batches prefabricating the components they will soon need. They also came up with a system in which all the parts for the dishwashers were stocked in open racks in the middle of the factory. Metal tags, green on one side and red on the other, hung on each rack, and the workers would flip the tags when they saw it was time to reorder, ensuring a steady supply. This was a big improvement on the traditional assembly line, in which dehumanized workers have no role in decisions regarding the production process.
What we see here are people electing to move from an assembly line to, basically, work cells implementing small batches of inventory with workers that are skilled in multiple areas. They even set up a “Kanban” system, though I doubt they knew it, where they had visual cues of when parts needed to be supplied.
Note that not once in these paragraphs does he mention the word “lean”. There was no implementation of “lean”, no “lean” or “continuous improvement” initiatives. This just seemed to make sense to the people doing the work and since they were allowed to do it - and knew enough about the overall process rather than just their small piece of the overall process (i.e. they were multi-skilled), they were able to see the obvious and execute it without all of the red tape - and get great results for the company.
He goes on:
The strength of these groups was their diversity. They included factory workers, engineers, office clerks, sales reps and executives. They didn’t have a formal head; whoever showed the greatest capacity to lead got the job, calling meetings and moderating discussions. In more than one group, a shop-floor worker guided professionals. Instead of a seniority system, or boxes on an organizational chart that guaranteed power, the groups were held together by a natural system of collegial respect.
Again, you hear this a lot in TPS. People are trained from the bottom up in the company and have skills in multiple areas. While there are “leads” that are responsible for a product line, everyone has the ability to lead when they are the most skilled for the job at hand.
The only vague reference to lean that Semler makes in this passage is the following paragraph, where he draws similarities and differences between what Semco did and TPS (though Toyota is not explicitly stated):
There are similarities between this system and the Japanese approach to organizing manufacturing operations, but also important differences. In our groups, younger members didn’t automatically submit to their elders. Moreover, once a team decided an issue, it stayed decided. There was no approval needed to make a change. Then again, there were no special rewards for new ideas. It was a spontaneous process; people participated only if they wanted to.
As I read this it really got me thinking. In most of the process agile / lean related books that I’ve read there seem to be a few common themes:
One more principle that I would add would be “tolerate mistakes”. Many of the issues that I’ve come across with other groups is that if they make a mistake they feel they will be punished. I’ve had great success with my team in articulating that I know mistakes will be made, but I want them to be made once, a lesson learned, and things put in place (usually automated) to ensure they won’t happen again. I’ve found that if people know it is expected that mistakes will be made, and everything doesn’t have to be perfect, they are more receptive to trying something new.
But I digress.
What Semler’s story shows me is that if people are given the freedom to work the way that is most effective, they will. More than that, if you invest in them with trust, they will want to do these things as their commitment to the company will obviously go up based on how they feel they are treated.
Semler uses a key phrase throughout his books that is repeated over and over. “Treat people like adults”. Semco, Toyota, Amazon and Google seem to do a really good job at this, as I’m sure most high functioning companies do. Read this article called The Google Way: Give Engineers Room and you will see the same concepts outlined in the excerpts on Semco that I have just written about. It seems to be a common theme.
So my real question. Is methodology and process really the answer, or is it deeper than that? Is it the way we treat employees that cause inefficiencies? If it is, if we took this base principal of trust and actually implemented it, would our employees come to the same conclusions as companies like Semco, Toyota and Google?
I think they would, because the principles and processes implemented by these companies are really just common sense without all of the complications of “process” and authoritarian management. They encourage workers to work outside their “box” and learn what they need to learn to be more effective. I would guess these employees feel valued, because they can constantly improve themselves rather than just “be the guy that puts the screw in the hole”. When you are allowed to improve yourself, your commitment rises to those who “allow” you to do so. What you wind up with is a highly efficient company that can change on a dime because people are allowed (and encouraged) to change and improve.
Most interestingly, the processes wind up looking “agile” or “lean”, without all the cruft of trying to follow a cook book.
Am I officially becoming a hippie, or does this line of thinking make sense? Let me know if I should go join a commune.
Tagged with: agile, amazon, continuous-improvement, corporate-culture, democracy, lean, process, semler, toyota, tps, trust
I’m reading The Machine That Changed the World : The Story of Lean Production by James P. Womack, Daniel T. Jones, and Daniel Roos. It is an extremely interesting book.
I ran into this small paragraph yesterday that for some reason stuck in my head as something important:
The truly lean plant has two key organizational features: It transfers the maximum number of tasks and responsibilities to those workers actually adding value to the car on the line, and it has in place a system for detecting defects that quickly traces every problem, once discovered, to its ultimate cause.
I’m telling you, the Poppendeick books are great, but there is nothing like going right to the source for an explanation of lean. I’m about 100 pages into the current book and I am absolutely fascinated at how much of todays current corporate structure (multi-level, many people with very specific task sets or responsibilities) is based on things that Ford and Sloan did with their companies.
In IT, this management style is manifested through all the different groups one hears about all the time from people in the field: Development, Infrastructure, Business Analysts, Quality Assurance. Each its own little silo, with its own responsibilities - and never should one group know how to do, or be privy to, the information in one of the other groups. Handoffs occur between the groups via very large documents.
Sometimes it goes further than that. I was talking to a friend once (who worked at another company, BTW) who told me about how their DBA’s were responsible for uptime and performance of the database and had decided that developers were not allowed to use ORDER BY clauses in their SQL because it effected the performance of the database. These developers were actually forced to sort their results within the application, rather than use the capabilities of the database, adding additional complexity to an already complex application. Worse, management seemed to buy into the decision, as I don’t think I would have been hearing about the situation if it was overruled. Ridiculous.
Another quote from the book, same page:
In old fashioned mass production plants, managers jealously guard information about conditions in the plant, thinking this knowledge is the key to their power.
Again, shocking how much of this mentality you read about in corporations not even connected to automobiles. This sounds like just about every company I’ve talked to people about (or worked at) over the years.
I’ve come to the decision over the years that ultimate transparency is the key to breaking down silos. It only breaks down your silo, but hey - thats a start, and at least you are setting an example.
Its definitely very beneficial, I’m finding, to read about things that are completely outside your profession to give you some distance from what is being taught. The lessons flow in easily this way, because you don’t have the predisposition that you “already know how things work”.
I recommend to anyone in IT to pick this book up. Its absolutely fascinating.
Tagged with: agile, Books, corporate-culture, lean, process, silos
Yesterday morning I found an article by a former employee who has recently introduced Scrum to his organization called Beef Up Your Scrum-Master Toolbox up on the Devx site. Since Doug started his blog and then subsequently left the company, I’ve enjoyed keeping up with his new adventures and I went right to the DevX site to find out what he had been doing lately.
The article is great, outlining the way that his team is accruing metrics for their Scrum teams using an Excel spreadsheet. However, one thing kept nagging at me the whole time I was reading it. The little voice in my head kept saying:
God, thats way too much work!
We recently moved most of our metrics, such as cycle time of items, to an automated process, eliminating about an hour worth of work from one of my managers that was doing it manually. As it stands right now, all of our metrics except one is generated automatically and posted to our internal intranet site. However, the system still isn’t perfect, as it requires development and QA to navigate items through a pretty tedious workflow to get accurate information, something that in the heat of a release is often easy to forget to do - just because its “extra stuff”. I know, because when I’m involved I often forget to update them myself. It seems to me that the real problem getting metrics is how much work is involved in making sure you get the data.
Doug’s article really got me thinking. How can we get the fine grained metrics he’s talking about, in a way where someone isn’t sitting in an Excel spreadsheet for an extended amount of time inputting data, and where all data is updated real time?
As I was thinking about this dilemma, I hit F12 on my Mac to look at the weather and I noticed this little box staring at me:
.. and then I started thinking. What about my experiences with Twitter could help with this problem?
I signed up for Twitter a few weeks ago after hearing Coté and Jason Calacanis mention it quite a few times on their respective podcasts and blogs. I wasn’t really impressed with it at all, only because I’m not willing to set up my phone to use it (I hate typing on a phone) and I don’t like having a separate web page up all of the time. Then I found the Twidget dashboard plugin for OS X and found that hitting F12 and typing a message was much more conducive to my working style than any of the above methods - it was less of an interruption.
Most of the twitter messages I post now are at home, where I have my F12 key handy and it takes little effort to actually post an answer to the question “What are you doing?”. Its effortless.
One of the adjustments we’ve made to our development process over the last few months is a logical reduction in the “types” of work we track. Rather than deal with “projects”, “change requests”, and “defects”, we are attempting to reduced these queues into something we call “open items”. For each change we want to implement, one or more “open items” are entered to address them. These items are then tracked for cycle time and completion status. Along with cycle time, we also keep track of the total number of “open items” we have that have not yet been deployed to production. These are reported on using your standard burndown chart that would look something like this:
For larger projects, the overall project is tracked on a Wiki, where we can both keep track of current tasks, and create technical documentation for the work we are doing as we do it. We’ve found that the documentation we create as we are doing the work is much more complete and accurate if we write it during the development effort than it is when we “go back later” to document it. A particular initiative is mapped to several smaller “open items” and scheduled for subsequent, incremental deployments (sprints, increments, whatever). In an ideal world, only the items that are scheduled for the next deployment would be worked on for the next deployment, but we aren’t there yet.
Tracking “open items” as a general unit of work gives us a start in creating a “pull system”, where the development team can work on open items as they come into the queue. The prioritization process, ideally, would be extremely light: pull things from the head of the queue, in order of criticality (Critical, High, Medium, Low), oldest first. This gives us a very simple ruleset to pull from (no prioritization meetings required!), and allows our business users to set a priority that can expedite changes if it is necessary. The defect tracking system, at this point, is a real time reflection of the work we have to do and the priority that the business thinks the work should be done in. This also allows “demand” to come in in a uniform way. The rules are simple and there is one entry point for all work.
On a weekly basis, data is pulled from the defect tracking system and loaded to our intranet database (a small data mart) where we calculate cycle times, and can graph the current rate of work.
The problem is that reporting is only a part of the problem. You can report on the data, but the data will only be accurate if you have an effortless way to update the items you are working on.
Its like Twitter - if I have a separate window that I have to keep going to in order to actually post something I’m less likely to make the effort - because I want to get done with what I’m doing.
Tracking work within the software development process is even worse if, in order to do it, I have to to look up an item over and over again to update it, and there is some complex workflow that it has to go through. Chances are, Ron is going to look like he is behind all the time, no matter how much work he’s getting done.
So what would a “perfect” work tracking system look like for me?
I’ve already asserted that I am more likely to Twitter if I can just hotkey to something simple, type something in and press my update button. I think the accuracy of measurements could be increased dramatically if we could have a system that took these types of things into account and reduced the context switches that the team has to go through in order to update status so that they can update real time and it feels like its part of the work that they are doing, rather than something separate. Its an added bonus if this feels like a social, fun activity rather than an extra set of tasks that need to be performed.
The way I see it, work can be entered into the system with the normal interface to the tracking system. This would give full visibility to everything needed to schedule work.
The new developer/QA interface to this system could look a lot like the Twitter widget above. Just a simple “What are you working on” type of interface that could talk to a network based API that, when a message is sent over, could search for a ticket number in the message and add a note to the item. A Subversion post-commit hook could use the same API to update the “code complete” status of the item (pulling the ticket number from the commit message), while a separate QA “widget” could have a pass / fail status pulldown that the QA team could use to signify the testing state.
As an added bonus, the deployment system could also use the API to let the QA team know the deployment status of the given item and whether it is actually there to test. This functionality, intersected with a “Pass” testing designation from the QA team in the production environment, could actually close the item automagically, which would reflect in your measurements.
The key to a system like this for me, is that if the updates feel like more of a social activity, and some of the obvious things are automated out of the process (like having to close an item once it is deployed to production and passed QA testing) the chances are high that you will actually get accurate real time status of what people are working on and the issues they may be running into. This is definitely some of what I am seeing as we use wiki software to track larger development efforts and as a manager the smaller grain information that I would get from a more “socially oriented” system would be extremely valuable to me just to know what is going on at any point in the process.
Now, this is just one view of the problem, from someone who is trying to lighten the large corporate process of getting work done. It might not work in every scenario or every environment - but I think it would be an interesting experiment.
I’d definitely be interested to hear other peoples opinions on this train of thought. Am I nuts or does this actually make some sense?
Tagged with: agile, continuity-of-experience, lean, metrics, process-improvement, reporting, twitter
I’ve started reading The Toyota Way: 14 Management Principles From The World’s Greatest Manufacturer by Jeffrey Liker. I’ve figured that as my curiosity peaks on Lean Development and Lean Principles in general, I might as well go to the source.
Chapter One opens with a quote from Fujio Cho, the president of Toyota Motor Corporation from 2002. I read the quote and thought I’d post it up here.
We place the highest value on actual implementation and taking action. There are many things one doesn’t understand and therefore, we ask them why don’t you just go ahead and take action; try to do something? You realize how little you know and you face your own failures and you simply can correct those failures and redo it again and at the second trial you realize another mistake or another thing you didn’t like so you can redo it once again. So by constant improvement, or should I say, the improvement based upon action, one can rise up to the higher level of practice and knowledge.
Toyota is thought of as one of the most process oriented companies around, and yet they still acknowledge that you do not know everything up front and build that into the process. A book that starts out this way has got to be one interesting read!
Tagged with: agile, corporate-culture, lean, principles, process, quotes, toyota
Last night I finally received my copy of Lean Software Development: An Agile Toolkit for Software Development Managers by Mary and Tom Poppendieck. I’ve actually had quite a few books on order and as they’ve been coming I’ve hoped that they were this one. Finally it got here.
I started getting really interested in “Lean Concepts” after reading The Goal by Eliyahu M. Goldratt and Jeff Cox, a very well written “parable” illustrating the application of lean principles and the Theory of Constraints to the manufacturing process. This was the first book in a long time that I was completely drawn into - so much so that I actually dreamed about the content after I had finished reading the book. Thanks to John Goodsen for recommending this book to me, among others, while attending a recent training.
This posting is not a book review of the Lean Software Development: An Agile Toolkit for Software Development Managers book - that will come later. However I did want to point out that rarely have I been sucked into a book as quickly as I have been with this one. I think that this is because what I’ve read so far maps so closely with the content of The Goal that it jarred me a bit.
Chapter One starts with the first principle of Lean Development. Identifying and eliminating waste. The authors define waste as “something that does not directly add value as perceived by the customer”. They also assert that “If there is a way to do without it, it is waste”.
Here’s the strongest piece of this argument, taken directly from the book:
In 1970, Winston Royce wrote that the fundamental steps of all software development are analysis and coding. “[While] many additional development steps are required, none contribute as directly to the final product as analysis and coding, and all drive up the development costs”. With our definition of waste, we can interpret Royce’s comment to indicate that every step in the waterfall process except analysis and coding is waste.
The argument that the authors are making really make sense to me. What pieces of accepted software development practices are adding “direct value as perceived by the customer”? Does the customer appreciate the long requirements and design processes that wind up feeding into a process in which documentation then has to be generated to change the design of the system after requirements have been frozen? Do they appreciate the fact that we have a “process” to document each change that we make, even though, when push comes to shove, the documentation is rarely looked at as often as the code is? Is the long, drawn out process we IT people use to try to keep our world under control adding immediate perceived value to our customers lives?
“Perceived value to the customer” is another reason why I have always been confused to see development teams put more value on being involved in “projects” than maintaining current systems, whether it be fixing reported defects or adding requested functionality to an application. In my mind, these smaller changes and fixing of defects found BY customers are the things that make the customers life easier and that they will get value from almost instantaneously upon deployment (not to mention that more times than not, they are “chunked” properly). Larger scale “projects”, mostly perceived by teams as “sexier” work, are essentially (in many cases) just a guess from the busines as to what might create value.
I can see from just the first part of this book that this is going to be a really interesting and valueable read. I think it will definitely get my brain working again - and I know there will probably be quite a few rambling posts like this one about thoughts I have as I go through it. This is an area of thought that completely excites me, mainly because there is so much waste in our industry (IT) as a whole. Its kind of nice to read books every now and again that confirm that many of the thought processes you go through in your professional life are not as crazy as they seem sometimes.
I found a really good podcast called The Agile Toolkit Podcast, in which the host, Bob Payne, attends agile conferences and interviews people there. Some of the interviews include people like Bob Martin and Mary Poppendieck among many others.
The most interesting show I’ve listened to so far is an interview with Dave Thomas (of Pragmatic Programmer fame) about agile development.
For me, I think I found it interesting just in the fact that it is nice to hear someone that has the same views on development issues as I do. I’ve always been a big believer that methodologies are limiting and that each methodology should be tailored to the project team. One part of the conversation that I found extremely interesting was when Bob and Dave were talking about the dogma attached to many of the methodologies.
Recently I had attended an Agile Development training in which the instructor stated that if you weren’t using all of the components of XP, you weren’t doing Agile development. A good point that Dave made was that as XP was being developed, the teams it was being developed with actually evolved into using all of the practices at different stages of their team development. In other words, they didn’t start using all of the practices specified in XP - because they didn’t exist yet. Dave makes the point that teams need to evolve into all of the practices - and that its very difficult to implement all of them at one time. I actually think that each team will be sufficiently different enough that you may not need all of the practices, but only a core set of practices. Bob Martin also makes this point in his interview and lists the minimum set of practices that include very short cycles, an open office (a room which holds the identity of the project), test driven development (both unit and acceptance tests). He also mentions that its extremely difficult to do test driven development without continuous integration, so there are other practices that will be necessary as you begin to implement the minimum set. I actually believe that source control and automated builds are another of the core minimum practices that should be put in place before anything else - but thats just me.
Another area that Bob and Dave talk about extensively is the necessity of developers to look at other languages in the industry other than the core language they use day to day. One statement Dave makes is that he looks forward to the day that developers refer to themselves as “developers” rather than “Java developers”. I wholeheartedly look forward to that day as well.
I’ve always enjoyed learning new languages. If you run through the articles in this blog, you’ll see that every time I find some language that I don’t know - and understand the practical reasons why they exist, the chances are I start working in it right away (most recently, this language is Objective-C). I enjoyed this part of the conversation a lot, because Dave articulates very well how learning new languages can give you new insights as to how to implement things in different ways.
I’m a firm believer that in software development, you have to have a pretty large tool box. The right tool should be used for the right job. This is why in many of the things I’ve done over the years, different components are written in different languages depending on what I am doing. A web piece might be written in PHP, scripts done in PERL or Python, while other components could be written in C / C++. I’ve made a conscious effort over the years to expose myself to as many different languages as possible.
In order to have the flexibility to use the right tool for the right job, you really have to make an effort to get at least a high level understanding of the different tools available and what their strengths are. Thats the really nice thing about the conversation with Dave is that he articulates the idea that you don’t necessarily have to be an expert in all languages, but know enough to use them and glean knowledge from them and their design.
I have found each of the shows I’ve listened to from the Agile Toolkit Podcast very informative and totally worth the time investment. In the very least, I want my teams to make this a part of their learning program moving forward. There’s no better place to learn about Agile practices than from the people right in the middle of it.
Tagged with: agile, development, podcasts, reviews
Coté from the DrunkandRetired podcast has written a very good article called Make the Iteration Fit the Deliverable, and Other Thoughts on Becoming Agile.
Key points I took away are:
In my mind, one of the misconceptions that people have when switching to any methodology is that there are a set of rules and procedures you must follow in order to be “doing the methodology”. I don’t think this is the case. The great thing about agile development is being able to adapt to the situation rather than be “stuck” having to follow a set of steps no matter what.
When I think about this idea, the quote from Bruce Lee about Jeet-Kune-Do, his martial art, always comes to mind:
Absorb what is useful, reject what is useless, and add what is specifically your own.
Bruce Lee was annoyed with traditional martial arts because it had “too much form”. The martial artist was unable to adapt to the unexpected. Rather he was limited by the forms he used while learning. (Before any martial artists start posting comments - this was Bruce Lee’s philosophy - not mine). Lee thought that the martial artist, given a set of tools, should be able to react naturally rather than having to think before responding.
I view development “methodologies” in the same way.
Things are going to happen. We have to be able to respond to them in order to be effective, and not get caught up in “how” we get things done. This to me is the power of the “philosophy” of agile development. It has nothing to do with iteration size or any of the other “guidelines” in the methodology.
Shorter iterations, however, do not mean that we do not design or plan during the process. It doesn’t mean working randomly. It means that we should do what is necessary in order to plan, design, and execute - no more and no less.
I think it would be more useful to explain Agile development as a philosophy in this way rather than a set of “methodologies”. People get caught up in “doing the methodology” rather than executing what they have set out to do. I have seen this time and time again as people try to change the way they do things. Agile development methodologies provide a set of tools, like TDD and the idea of iterations. Its up to you and the needs of the project to decide which tools are appropriate to get the work done.
Anyway, I digress. Coté has obviously set off a whole train of thought in my head this morning with this article. Perhaps it will do the same for you.
Tagged with: agile, development
