Matt O’ Rama

Mad Science Alphabet Blocks

Mad Science Alphabet Blocks | Xylocopa.

Mad Scientist Alphabet Blocks

I know what the Grommes kids are getting for Christmas now! Click on the picture and check out the detail on these, they are 100% awesome.

Using evolutionary algorithms to make a walkthrough for the light-bot game with C# at Chris’ Babbling Blog

Using evolutionary algorithms to make a walkthrough for the light-bot game with C# at Chris’ Babbling Blog.

This is cool. I’ve always been very interested in genetic/evolutionary algorithms and one of the ways I’ve wanted to practice is writing a simulation of a creature that figures out how to walk a path. This guy used this same idea to solve a real game. Nice stuff, although I wish there were some more technical details. (He does provide the code though).

A cool thing he sees is something a lot of GA research finds, that the algorithms often evolve unintuitive or extra-complex solutions. Just as in real life, evolution is only interested in solving the problem.

My First Agile Project Part 6: The First End Of Our Project

Originally published on AgileSoftwareDevelopment.com

Picture courtesy of Photo Mojo@flickr

Welcome to Part 6 of my team’s first adventures with Scrum. If you want to start from the beginning, see the table of contents at the bottom of this article. Each part stands alone though so don’t worry.

In this part of the series, I’ll talk about what we did when we approached what we thought was going to be the end of the project. As it turned out it was only the first deadline that we would miss. Near the first deadline we went off of doing Scrum sprints and planning in favor of fixing bugs as they came up in testing. This cost us a lot of team cohesion and focus. Read on to hear about what led us to missing the deadline, why we went off Scrum and how we got back on the right path. As usual, I hope our story helps illuminate some decisions teams make with the best of intentions that can lead to unforeseen negative consequences.

—

During our Inception Phase the vendor whose billing system we were working with came up with a development model (basically a complicated spreadsheet) that was a SWAG at how long they thought this project would take, based on how many people we had on the team and a rough estimate of how much work we had to do. The model showed we would need 9 month-long development sprints and 3 integration testing sprints. As we neared the end of the development sprints, we knew we would need more time so we said at first we would take the first testing sprint and finish up the last development. Of course we needed more than that sprint so we pushed back testing another sprint leaving about a month for final testing before go-live.

The idea with leaving the integration / User Acceptance Testing to the end was that we thought unit testing and developer testing would be enough to flesh things out during development enough that we would just be squashing bugs during the final sprints. There were a couple of reasons this didn’t work like we thought it would but that’s a bigger topic that I’ll be going into in more detail next time.

Going off Scrum

Once we came to what we thought was going to be the final push to going live, we decided not to do structured sprints and just hit the bugs that came up in testing as fast as we could. We didn’t think we had enough left to justify doing a sprint. So for a few weeks we just followed the testing team, trying to take care of bugs as they came up. The more we went along though, the more apparent it became that testing was uncovering a lot more work than we thought it would. I don’t remember when we finally admitted that we weren’t going to make the deadline but it was after we should have admitted it. We were in denial.

After the first deadline passed and we were still working hard, as well as uncovering lots of work still to do, we should have gone back to doing sprints. I think we were still partially in denial as a team, thinking we would finish up soon and be able to go live. Team denial is a strong force and I think the only way to get around it is to make sure everybody speaks up. The whole team wasn’t 100% in denial about the amount of work we had but each person’s low level of unease should add up. We all had thoughts, we just didn’t listen to each other enough and put things together. It’s everybody’s responsibility to speak up and if they have reservations about something on the project they need to make sure everybody else is listening. If we had really been paying attention to our data conversion team, for example, when they said they weren’t comfortable with the initial deadline we would have pushed it back before hitting it. And we should have been more honest about the amount of work we were still uncovering. It’s been almost 6 months since the first deadline passed and we’re still finding things, which tells us something about where we really were back then (luckily now the new findings are going on the post go live backlog).

Loss of Team Unity

Since we thought we would be going live “any day now” and we weren’t doing sprints, we lost some of our cohesion as a team. One good thing we did was keeping our morning Scrum meeting, although it morphed a little. When we were going to be hitting bugs in preparation for go-live, we decided a sit-down meeting that might last more than 15 minutes was appropriate since we would be discussing bugs more and making first passes at solutions during the meeting to expedite the work. These meetings meant we all still knew what everybody was doing, but not keeping a taskboard or doing demos ended up being much more important to team unity than we thought. It’s a subtle thing that didn’t really realize until recently but looking at everybody’s work as a team effort and showing your stuff at the end of the sprint really pulls people together. By this time we all had areas of expertise in the product and we were working largely on fixes and additions to those things independently of each other. We still worked together a lot but had we not been a close, friendly team I think we would have drifted apart a lot more.

Sprinting Again

After a few months of what I called wandering around, instead of sprinting, fixing bugs and taking care of issues from testing, we’ve recently gone back to sprints. We’ve all been broken of our delusions about going live “any day now” and know it’s going to be some time still. I think it’s a testament to our team that we’re not all disillusioned and we still have our team morale. We’ve gone back to 2 week sprints, which helps planning meetings not take all day and gives us much more flexibility. Before we went back to sprinting, we had a big meeting with our vendor to work out a plan and a new model for completing everything. This model showed we were going to be at this for quite awhile longer. Once we started sprinting again however, we blew this model away with what we were getting done and the finish line looks closer than ever. Going back to the Scrum processes we liked has really re-energized the team. It’s very tempting to try something different when things get tough but we should have resisted the urge to move away from Scrum. Not doing sprints when you’re about to go live and are squashing the last-minute bugs is fine I think but we shouldn’t have left the path as early as we did. We’re all glad we’re back on now though and moving forward faster than ever.

Next time, I’ll be talking more in depth about testing. Why unit testing isn’t enough, how we did user acceptance testing, and why we should have started our testing team going much earlier than we did. Stay tuned and thanks for reading!

My First Agile Project Series
Part 1: Doing 80%
Part 2: Inception & Planning
Part 3: Viral Videos and Bad Jokes in Scrum Demos
Part 4: How to lose credibility and jeopardize your project with lack of management buy-in
Part 5: Our Top 5 Agile Mistakes

Typing speed test

Speedtest – how fast are you?.

Steve Yegge had a post the other day about learning how to type faster, a skill most programmers probably ignore. I know I did. I’ve never really properly touch-typed. I don’t use the home row or any of that stuff but I don’t look at the keyboard either and I type fairly quickly. I knew I could get better though so I was happy when I found this test. It doesn’t take too long and it seems like just doing this a few times a day will help. For the record right now I’m hovering right around 52 words per minute. I’m hoping to get up to 60 average in a month and see how hard that is to achieve.

My First Agile Project Part 5: Our Top 5 Agile Mistakes

Originally published on AgileSoftwareDevelopment.com This has been, by far, my most popular post on ASD. People like lists, and mistakes.

Picture courtesy of DWQ Online@flickr

In the previous parts of this series (1, 2, 3, 4), I went into a lot of the initial issues of how we ran our project and some of the things we did wrong. For Part 5, I’m going to focus on the 5 big mistakes we made in the project before I move onto another phase of the series.

For some background, the project was to integrate an off-the-shelf (but highly customizable) billing system to replace an aging custom Oracle forms app. This was our company’s first foray into Scrum and Agile as well as the first Agile experience for most of the development team. After we sailed past our second management-imposed deadline I started thinking about how we could correct our process for this project and the next ones we do, which led me to write it all down in this series. Keep reading for the Top 5 mistakes we made (in no particular order). Where possible I also give some detail about how we’ve worked on correcting our process. I hope this list helps you avoid these mistakes and make your own all-new ones. :)

1) Not tracking backlog
This was by far our biggest mistake I think. As I mentioned in Part 1, we failed to keep track of how much work we were adding to our backlog as we went along. We all assumed that since we were being Agile, we would add work as we fleshed out requirements and learned new capabilities. The problem came with not tracking how much work we were actually adding and comparing it to how much we were doing in each sprint. Without doing this, another of our mistakes – Scope Control – became much more of a problem than it might have. Not reassessing the backlog periodically meant we didn’t know how scope creep was really affecting the project, which meant a downward spiral of allowing the scope to creep, not knowing the effect of the new scope, allowing scope to creep more, etc., etc.

Since we realized a few months ago we should have been doing this, we’ve been keeping much better track of what gets added to the backlog and making sure we’re being as ruthless as we can about moving features to Post Go-Live and only working on important bugs. As a result we have a much better view of the finish line of the project.

2) Too much planning upfront, not enough later
In Part 2 I talked about our Inception Phase where we had our initial meetings to get a rough estimate of the integrations we would be doing and business requirements from other departments who would be using or relying on the billing system we were building. The mistake we made here was in the amount of requirements gathering we did and how we did it. We took a “snorkel, not dive” approach where we didn’t get too many details, which is a good approach. But we also made requirements documents complete with flow charts of integration program flow. These documents took much too long to write and didn’t end being used by the developers at all. Once we started sprinting and doing the work, we re-learned, or un-learned, pretty much everything in the documents but it all made more sense since we had the context of our code to work in.

Obviously not getting deep technical requirements up front means you need to get them later and this is part 2 of this mistake. We should have made it our practice to start getting requirements for the next sprint so we could start working without waiting. As it happened, our Product Owner/Subject Matter Expert was busy enough that we spent a lot of time waiting for requirements before we could start.

3) Boring business people with technical demos
I was tempted to make this mistake “Didn’t get management involved properly” but I think the early mistake was our demos. As I said in Part 4, we demoed everything we did that sprint, including infrastructure and behind-the-scenes integrations. This is good for the team to get to show off but it’s boring and turns off the business people and management. Once you’ve turned people off the process, it’s hard to get them back into it when you need their input. They’re still going to give you input, it’s just going to be later than you expect and usually after you think you’re done with the thing they should have seen 3 sprints ago.

Making the demos boring for the business folks also meant we failed to properly engage our upper management. We ignored this problem for too long and once they started getting more engaged, it was after the deadline had been blown and they weren’t aware. Not a good position to be in. Having more business focused demos wouldn’t have guaranteed anything but combining better demos with an insistence that the business / management people attend would have helped immensely.

4) Not estimating product backlog items
Part 1 of this series talked about how we were Doing 80% of Scrum. What we missed was a chunk of related, but separate processes around estimating and planning the backlog. This mistake is tied to a couple of the others because of this but we committed them all separately so I’m listing them separately. Our product backlog was basically just a list of work we had to do. Our Product Owner prioritized her most important items before the sprint planning meeting, then we moved the items we were going to do over and estimated them then. We didn’t estimate the backlog items ahead of time so she could know how hard we thought things were or so we could know how much more work was ahead of us. We had rough ideas that Integration X would take longer than Configuration Y but without real estimates, we couldn’t start harder things earlier in the process or push things that were hard and low priority off until later, both of which are vital for planning.

Since we’re in bug-fixing mode right now, we’re struggling with fixing this mistake more than with any of the others. I got a bunch of Planning Poker cards from Mike Cohn at Agile2008 and we’ve been trying to use those but it’s hard when the issues you’re estimating are bug fixes that only the person who worked on it originally knows about. If you have any suggestions on this, I’d love to hear about it in the comments.

5) Scope control
Not controlling the scope of the project, combined with our lack of estimates and not monitoring the growth of the backlog, constitute by far the largest cause of problems with our project. If we had been estimating our backlog items it would have helped us track the amount of work we had left, which would have helped us know when and what to remove from the backlog and keep to the next release. Since we didn’t do any of those things, the backlog grew too much, which pushed back the set of testing sprints we had scheduled, which pushed back the work that testing revealed, which put us in the position we’re in today, with a late project, lower morale, and lost credibility with management.

One factor I’m struggling to figure out is that the nature of our project seems to limit our ability to control scope. We’re doing an integration of a new billing system to replace an old system. This means there’s a set of existing functionality we have to duplicate in the new system that can’t be pushed to the next release. Also, being a billing system you have to get things right the first time. You can’t send out a partial invoice, for example, just because doing the whole thing will take too long. But I’m sure everybody thinks there are exceptional things about their particular project so I don’t want to fall into the trap of saying we can’t learn anything about how to control scope because our project is the exception. There’s always something that can be pushed back and if you’re doing the right things with estimation and monitoring your backlog it makes finding those things much, much easier.

We’ve been in bug-fixing / testing mode for the past few months and have tried much harder to keep things out of the backlog if they didn’t 100% need to be there. There’s still things going in, but now we make sure each thing is absolutely needed for go-live and everything else goes on the list for after.

Mistakes != Fail

We’ve made mistakes on our project, for sure. We’re not done yet so I’m sure we’ll make more. But we’re learning, which is the important thing for our next projects. As a team we’re all committed to Scrum and being Agile so we’re going to make the effort to improve our processes where we think it’ll help. The more technical-oriented aspects of Scrum are incredibly appealing to us as developers. Keeping a backlog, committing to work, keeping interruptions to a minimum, sprints; all of this we’ve done mostly correctly and we love how it’s worked over the past year and a half.

I’ve spent the last 5 parts of this series talking about the basics of how we ran our project and the lessons we’ve learned while we chugged along sprinting. Next time, I’m going to start talking about what happened once we missed our first deadline. We changed our process, halfway-unknowingly moved away from Scrum, and suffered for it until recently when we did a reset of our process and got mostly back on course. Stay tuned next week and as always, I’d love to hear any comments you have on this series. Thanks for reading.

“Automated story-based acceptance tests lead to unmaintainable systems”

Projects where the team directly translates story-level acceptance criteria into new automated test cases set themselves up for a maintenance nightmare.

thekua.com@work » Automated story-based acceptance tests lead to unmaintainable systems.

One of the big things I’d like to do with our next projects is some kind of automated acceptance and UI regression tests. I’ve looked at things like Selenium and a little at FitNesse but I’ve worried about things like this article talks about, the tests becoming a maintenance challenge by themselves.

ScrumShocked writes about my articles

I've been following a very interesting series of posts (starting here) from Matt Grommes detailing his experiences managing his first agile project

ScrumShocked.

This is cool. This guy found my articles on AgileSoftwareDevelopment.com and writes about our experiences compared to his teams. This is kind of like being recognized on the street, awesome but strange. I like that people are getting something out of my articles though.

My First Agile Project Part 4: How to lose credibility and jeopardize your project with lack of management buy-in

Originally published on AgileSoftwareDevelopment.com

Picture courtesy of shaz wildcat@flickr

Welcome to Part 4 of My First Agile Project. If you haven’t read the others in the series, don’t worry. Each part should stand alone. If you want to read the whole story though, the other parts are available: Part 1, Part 2, Part 3.

In Part 3 of this series, I talked about our demos and I mentioned how they unfortunately turned people off the demo with presentations about mostly behind-the-scenes features. My first instinct was just to say that those people lost their right to have input since they weren’t involved in the demo. In reality, the business world just doesn’t work like this, much to my chagrin.

What happened was the people who didn’t care to come to the demos just came later with changes and “suggestions” for new work we had to do. Also, since a lot of the ones who were supposed to be coming were upper management, their lack of knowledge about the Agile processes we were using led to a lot of problems and misunderstandings later in the project. Read on to hear more about how we failed to get our management on board as much as we should have. Hopefully you can avoid that mistake and some of the problems we’ve faced.

Consequences of Boring Demos

Our tendency on the project was to demo everything. We wanted everybody to get credit for their work and the audience to get to comment on everything. We wanted to be transparent so we showed everything, even though we knew it was sometimes boring for the business people in the audience.

At first, I didn’t think this was as big of an issue as it really was since I thought people understood the process and would pay attention because this was their chance to give feedback. Instead, we got comments from people like “I’m not going to waste 2 hours sitting there listening to IT clap for each other.” This wasn’t to our faces but behind our backs when higher-level people were asked to explain why they didn’t bother coming to the demos.

I’m not sure what the answer is to this. It’s valuable to have these kinds of demos so people can show off their hard work and get the team on the same page. And it does give people the chance to give feedback, but in practice, demos for stuff that happens behind the scenes can turn people off the whole process, which we learned can be worse than just not getting some feedback. Once somebody decides they’re not going to come to the demos any more, it’s hard to get them to reconsider when you get to the stuff they should be seeing. What ended up happening was that we would get feedback many sprints after we thought we’d “finished” some piece of functionality as word got back to somebody about what we’d built. We had to do a lot of rework due to this. Even things like finding out later what paper we were going to use for printing invoices came back and caused rework.

Getting Management Involved

Beyond rework, not having management involved is an even more important consequence of turning people off the project. At our company, managers and directors do a lot of the work so we invited a lot of upper level people to our demos. Getting these people on board and understanding the process is more important to an Agile project than I originally thought. Our project was the first exposure that our company had to Scrum and Agile processes so we tried to explain what it all meant so everybody would know. It was clear early on that the Agile ideas weren’t being understood at the higher levels of the company so at our second or third demo I gave a small presentation about the Scrum process and thought that some upper management people would be there to learn about it but they didn’t end up coming. We tried as we went along to get everybody on the same page as well but we mostly used the demo as the communication channel so with people not attending the demos knowledge of the problems didn’t get out there like they should have. In retrospect we were not as forceful as we should have been in getting the information out there. You just can’t expect that business people will be interested in something like Agile without a lot of hand-holding about it.

When we got down the line and were in trouble with problems on multiple fronts our management just saw the project was late and we were responsible. The clash between how a company wants to work with big projects (set deadlines, set budgets, etc.) and how Agile deals with big projects is real and if you don’t work hard to get people to understand the differences, you can’t come in after the deadline has been blown and expect to educate anybody. You need to start early with planning and estimating, make sure everyone involved knows where the project is at all times, only then can they reconcile how the project is going with the budgets and the deadline. See Part 1 of this series for more on the problems we had with not estimating and planning correctly. But you can do all the planning in the world and if the information isn’t getting to management or if they don’t know what it means, it’s just noise and people will ignore it, to your peril.

If we’d been communicating all along, we’d be in a much better place. As it is now, I can see that we as a team have lost a lot of credibility and I believe our ability to continue using agile processes is probably in jeopardy.

Team Responsibility

We can argue that a lot of the problems we’re having are because of bad 3rd party vendors, complications in the product we were integrating, problems converting data, etc. but at the end of the day we’re a team and this is our project so the responsibility is on all of us. Agile isn’t just about getting management off your back or freeing the team from over-documenting. It’s about the team taking responsibility and part of that is not giving up power over your project any more than you can help. Good managers don’t just shrug their shoulders and let things happen to their projects and that includes self-managed teams. We should have insisted that certain people attend our demos and keep up on our project status just like we insist that our fellow team members don’t slack off and endanger the project. That takes some courage, to be sure, but it’s important. When you’re building a product for internal use, the users and management are a part of the development process just like the development team is. This is a change for a lot of people but the team has to insist that everybody participate if the outcome is going to be a good one. We’ve certainly learned this lesson the hard way but sometimes the hard way makes the best teacher so we won’t soon forget.

If you’ve had issues with getting management involved on your project, please share. I think it’s in the nature of most developers to not place the importance on management participation that the issue deserves so if you’ve learned this lesson, or if you think I’m putting too much emphasis on it, I’d like to hear about it. Thanks.