How can we plan sprints that accommodate both development and applied research stories with success?

I first published an adaptation of the SCRUM guide several months ago. I was planning on following that with a series of posts sharing some tips and other experiences I learned during the last 4 years using and developing this methodology. However, during this time, other things happened in my life, I talked about this at HackTown 2024, in Santa Rita do Sapucaí, MG and I ended up not having time to continue writing about this. Now is the time to continue!

In this post I’m going to share some experiences on what I learned about when planning sprints in research projects that had both development and research stories.

Sprint Planning

Although, the guide starts with the Sprint, here I will begin talking about the Planning: The Sprint Planning is the process where we define the work to be performed during a Sprint. This is a collaborative plan that is built with the developers, researchers and other stakeholders.

We usually start by looking at the Sprint Backlog and selecting potential Research and Development stories to be added to the sprint. However, there are some interesting topics that must be addressed during the planning, such as:

What is the value of this Sprint?

At this point, we must identify and define what is valuable about this sprint? What is the goal that the group is trying to achieve during this specific sprint. With this, the team defines the Sprint Goal, that will be used on the next question.

What can be Done this Sprint?

This perhaps is the main question everyone has: which research and development stories can we squeeze in this sprint in order to achieve the Sprint Goal that was defined?

This process is even more difficult for a new team that does not have a previous experience working together and thus doesn’t know what is the actual team performance.

There are several ways of defining effort or time, or story points to each story. And there is no perfect method. However, the bigger problem here is that usually, in software development for example, we inherently know what the effort of a task is from our previous knowledge: we know that a new page will take X hours to design, Y hours to develop and Z hours to test and validate; in research, this is not as clear.

In a research scenario, we may face a task that asks for a literature review of the latest work in a specific topic (an example that many of us may face as it’s rather common). This is a tricky task because it is difficult to know how much time we will need to research a new topic because we don’t know how many papers and books we may find in the topic, therefore, it’s quite easy to over, or more realistically, undershoot the actual effort needed.

In these and other cases where it’s difficult to determine the actual effort or where the effort is not clear, that is, where we could end up staying one or 6 months on the same subject, we need to define stop points and other parameters that limit our research: we may obviously define the timespan where we will search the literature, but more than that we must define the amount of time that we are comfortable investing in a line of research to determine whether it is relevant or not.

But what does that mean? It means that maybe a line of research may bring us useful results, but the effort to gather information in order to actually benefit for the results may be too much for the time of the project at hand. That may be because the learning curve is too steep, or this line of research is too complex or other reason and we may need to abandon it and search for a more feasible alternative.

Therefore in these cases during the planning, we must define not the effort that we know it is needed to complete the literature review, but rather, how much effort the team can spend on that task. It can even take more than one sprint, that is not a problem, because, in any way we may have preliminary results for the Sprint Review. But we must define how much time we have to spare and invest on that line of research.

This definition must be clear to all stakeholders and they must be regularly updated on the progress being made during the daily meetings and review meetings.

At the next planning, the same story may be brought back to discussion if needed for more or less time to be invested, but the team must define a limit in order to prevent the research from entering a limbo state of infinite research.

How will the chosen work get done?

This is the last topic and it’s where we enter the specifics of how we will attack a story so that we can complete the work. Here we check the Definition of Done and plan actions that will enable us to achieve it.

This is also the moment where we may define the research parameters for a particular task and where we create tasks that are small enough to be divided between members in order to parallelize the work or make progress tracking easier.

Rinse and Repeat

This process is repeated for every research or development item in the backlog and at the beginning of every Sprint. It’s important to note that the longer the Planning takes, the more time from the actual research and development is taken, also, the more time in a meeting, the bigger is the risk of people’s mind wondering about and the effectiveness of the meeting decreasing.

Therefore, it’s important to keep Plannings short and, if necessary, schedule other meetings prior to the next Planning to refine the backlog and discuss future work.

Also, the Scrum Master and Project Owner do have the last say in any topic, however, harmony and agreement between all members and stakeholders is extremely beneficial for the success of any project.

That’s it for now! If you have any comments, leave here and I’ll try to answer. If you wish to share an experience that you’ve had, you are welcome to post as well and any discussion is valid in order to improve this methodology!