In Part I & Part II we talked about the overall structure of the PER process & the information we collect. In this post we will talk about the meeting we use to wrap all this up. Any time we have a service impacting event we have a PER meeting to discuss the event and gather all that information we discussed in Part II. Read on for details about that meeting.
A note about facilitating a PER meeting
At Rally we are big fans of meeting facilitators and we even have a group of people who have been trained to facilitate meetings for teams they aren’t on. Often meetings at Rally aren’t run by a manager or person on the team but rather by a facilitator who is asked to come in and facilitate. The facilitator can have a large impact on the outcome of the meeting, both in the quality of the result as well as whether or not you finish on time. We could write a whole post on meeting facilitation but here are a few things to keep in mind when you facilitate a PER meeting:
- Avoid coming up with a solution to the problem. The PER meeting stops at identifying what the next action for a given delta is.
- The Facilitator should represent the group. This is why a 3rd party facilitator is good – they don’t have an opinion about what is being discussed. As a member of the team, make sure you get the groups permission when you add something to the document & allow the group to drive the decisions. Note: this is hard.
- Control cross-talk. Discussions like this can get pretty passionate and folks will start talking over each other – as the facilitator you need to make sure everyone has a chance to talk. Sometimes your most timid members will have great ideas – you want to hear those.
- Keep it on time. These meetings are draining if they run too long – if you need to do some research & come back for a 2nd meeting do that, don’t hold everyone in a meeting trying to get answers you don’t have.
Scheduling the PER Meeting
We have a rule that PER meetings must be held within 24 hours of an event to make sure things are fresh in folks' minds. No, we don’t schedule them on weekends for end-of-the-week events – we do them on Monday. For most events we'll schedule an hour and we can typically wrap up a PER in that time. For larger teams you may need a longer meeting and for any large event where you have multiple teams you may have to extend the meeting.
The PER meeting should include anyone who was involved in the event or needs to be present to come up with corrective actions. For problems where the component impacted is maintained by Operations, we will typically keep the meeting constrained to Ops, but for events where an application problem was involved we include a broader audience.
Folks at Rally who might get called into a PER meeting include
- Operations – Familiar with the event and typically the ones who responded to it.
- Development – Familiar with the applications & service as well as what might be tweaked to improve things.
- Product Owner – Able to prioritize important changes within Development
- Customer Support – Responsible for communication with customers during an event
For significant enough events you might even have folks from Marketing and Legal involved. Let’s hope that’s not the case – but don’t exclude anyone who would be able to help improve things.
Outcome of the PER Meeting
When we fire up a PER meeting we have a few specific goals:
- Make sure we have an accurate and complete timeline of the event
- Identify the most complete list of pluses and deltas possible
- Identify corrective actions and assign ownership to someone
- Agree on the overall event metrics
That’s it. Now granted, getting those goals completed can be a long road sometimes. In the documents linked below we have included a detailed meeting agenda as well as the items that we cover for meeting preparation. I would encourage you to have a look at those documents to get an idea of what items we cover in the meeting & what the structure of the meeting is – there’s not a lot of reason to cover that here.
For folks who may not already be doing this we encourage you to find your own balance in this process. The best process is one that you actually use. Do not create a process that you avoid because it’s too much work or too heavyweight.
We’ve made our own documents around this process public to help you with some examples. You are welcome to copy as much of these as you would like and make them your own.
I appreciate you sticking with us to this point. If you have feedback on this process or ways that you’ve found which work in your environment please share them in the comments – we want to know what others are doing too.