How To Manage Standup Meetings As A Machine Learning Team Leader
How To Manage Standup Meetings As A Machine Learning Team Leader
Learnings from being in thousands of standup meetings. Both as a participant and as a manager.
Hello readers,
Welcome to Infinite Curiosity. I recently came across an interesting question — How should I run standup meetings as a manager of an ML team? I decided to tackle that question in this post.
In this post, we’ll talk about:
How to recognize bad standup meetings
Standup agenda for ML teams
How to plan ongoing and long-term ML work
How to manage backlog
If you have a question, submit it here and I’ll get back to you with my thoughts. Subscribe to this newsletter to receive it in your inbox every week:
See the image above? This is what engineering standup meetings looked like in the 19th century. Or at least that’s what DALL-E thinks.
In an engineering standup, everyone gets a turn where they're supposed to answer three key questions:
What did you do yesterday?
What do you plan to do today?
What roadblocks do you have?
But how does that translate to a machine learning team? In this post, we'll talk about how we can take this format and adapt it to a team of ML practitioners.
How to recognize if your standup meeting has gone bad?
To adopt a certain famous quote from Anna Karenina:
All bad standup meetings are alike, but every good standup meeting is good in its own way.
ML work has inherent uncertainty associated with it. You should learn to get comfortable with it. If a standup consists of people who are not ML practitioners, they'll have a hard time understanding why ML practitioners can't commit to a date. I've discussed it in more detail in one of my previous posts.
Here are the characteristics of bad standup meetings:
Meetings are long and tedious
The meeting agenda seems unclear
There's that person who rambles on forever (if it's you, you probably don't know that it's you)
There's that person who provides a single-sentence update (if it's you, just know that it sticks out like a sore thumb)
Someone using this time to do a deep-dive on a new ML algorithm
Someone using this session to solve problems
People present scattered thoughts
People don't listen to what others are saying. They just check out after their turn is over. If the standup is happening over zoom, people start checking their email when others are talking.
You don't have to be the team manager to ensure that the standup meetings are productive. If you're part of a standup meeting, you should take the lead on making these meetings productive.
What should the standup agenda look like for an ML team?
If you're managing an ML team, you should aim to enforce structure and brevity. Ask the team members to send super-short bullet points before the meeting everyday. Getting people to write things down forces clarity.
Standups are usually 15 mins long. You should discuss the following items:
Is everyone aware of everyone else's current tasks?
Are you on track to complete the outstanding tasks?
Do you have the data you need to complete those tasks?
What ML experiments are you running right now?
What personal blockers do you have?
What dependencies do you have on others that need to be resolved?
How to plan ongoing and long-term ML work?
Daily standups are for immediate needs. But at the same time, you need to plan long term work as well. This is usually accomplished in weekly or biweekly sprints. You should figure out how to slot in the following work:
Operational needs of existing customers: This includes monitoring ML models, retraining as required, and ensuring that the model is still meeting the customer needs.
Deploying for new customers: This includes onboarding new customers, making sure the models are trained as needed, and making sure the users know how to use your product.
Features that will make the product better: This includes model performance, latency, or any other feature that will make your customer like the product even more.
Running experiments: These are ML experiments that might move the needle, but take longer to materialize. It can include testing new algorithms, improving existing ones, speeding up data processing, or reducing the compute power needed to run the product.
How to manage the tasks that don't get completed within a given sprint?
These tasks are usually put in the backlog. Having a healthy backlog of items to work on is a good thing. If you finish your tasks early, you can always work on backlog items. But backlog needs to be groomed in a disciplined way.
You create a list of items to work on, but some of them get pushed to the next cycle. This backlog is good to have since it helps you plan ahead. But you should set a timeline for how long you're going to let an item languish. The goal is to make sure the backlog list maps to your team's current capacity.
If the backlogs get super long, it's not great for morale. Not having any backlog is not great either. Make sure you're honest with yourself on what the backlog represents for the company and how it moves the needle.
If you are getting value from this newsletter, consider subscribing and sharing with your friends: