Give this doc a home

Keep your docs nice and tidy by moving this doc to a folder.

PT Estimator

Links

Old Designs

Similar Tools

Scrum Poker

FirePoker

PlanItPoker

Slack tools

Poker Planner

Standuply

Poker Bot

Story Poker

Team O’clock

ScrumPear

PM Interviews (Full Notes)

⬇️ Questions	Steve (10:00)	Lauren (11:00)	Gavin (11:30)	Furhana (3:30)	Paul (4:00)
Sprint Planning
Can you walk us through your last refinement/sprint planning?	1.5 week before project go to Jira Stories that will be brought into next sprint = Pointed (Tech Lead) & Hope to be Pointed Tech Lead asks devs for feedback, not much is given (may be in DMs?) Devs can look before coming up with their estimate Sprint planning: move everything discussed with tech lead and that was pointed only to sprint Will explain all stories, usually devs have not seen stories	Uses sprint planning to get devs and clients to agree Before → write tickets in icebox, clear reasons why work is necessary acceptance criteria inside ticket, what a feature is expected to do avoid implementation details Once estimated, moved to backlog Make sure people are comfortable w/ amt of points during Sprint Planning Dependent on team	Two meetings → backlog planning and sprint planning 2 week sprint Sprint planning alone is not enough Backlog Meeting: estimate stories that aren’t, get on same page, coming up in next few sprints Design artifacts included client included Sprint Meeting: commit to certain stories Tickets and all details should be understood/included	Features in the backlog, go through stories and point them Collective estimation No velocity Make sure number of points is realistic for delivery Backlog grooming: reassessment, look at stories that might have been too big Agreement of what to do during a sprint is done before the sprint planning takes place Points are confirmed during sprint planning	Work with customer to get roadmap for the next couple months Understand what the process looks like Then start writing out the stories in the icebox which serves as a place to write initial ideas, for a few days Move tickets that feel ready to review before planning meeting ⭐So people can read, but they rarely do Prioritize within grooming By the end of the discussion everyone kind of knows the points for each ticket
How do you estimate stories with your team?	If it’s pointed, will bring it into the next sprint (Tech Leads point), stories that hope to be pointed When tech lead asks developers for feedback not much is given Switched from group-estimate, realized it wasn’t a great use of time Too many follow-up questions Benefits: less pressure	Go through stories PM wants estimated. Group estimate or comes from most likely person to do work E.g. newer people might find it more difficult than experienced devs Will ask for opinions before story is up for estimate	Sometimes team is uncomfortable estimating at certain points Might not have time, full criteria explained, etc. (aka. lots of TBDs) Gavin thinks tools like planning poker are important to allow conversation and differing opinions. — in order to gain consensus.	Engineer who takes the story points them based on how familiar they are, how complicated it is, how much integration is necessary Designer looks at how big the design portion of the feature is and helps with pointing Talk offline about the estimate, then confirmed during the spring meeting Compared to previous tickets and estimates Senior engineers point stories accurately since they have more experience Small projects: Pointing Poker in person, but people gravitate towards the most senior person Larger projects: tech lead does the story pointing and asks for feedback	Sometimes play poker, other times just ask for suggestions (go one by one) Just picks a point value if people are evenly split (2 vs 3) Small point differences tend to have little to no change in priority
Do estimates change? Do you document when point estimate changes?	When estimate is not accurate, story is repointed before the story is taken discussed in comments in Jira	If the points aren’t right, then change them use as reflection of the person doing work/history Might assign someone to a backlog, but then assigned to new (changes velocity/size) Add as comment in PT (when remembered) Actual + Reasons	Prior to a sprint commit Track comments, don’t have to update if turns out to be larger Talk about changes in sprint retrospective Size doesn’t matter, just way to be able to estimate delivery	Estimates change when stories become too big Changes for multiple reasons: Dependent on another feature Blockers - for example Trinity may not be done with their work in time The tech lead doesn’t have time to accept certain things that are finished so something moves into the next sprint delivery Changing factors are external because the engineers are mostly seniors and know how to estimate stories	Estimates never change if new requirements are discovered, it becomes a new ticket Value of estimate is left the same even if it discovered to be inaccurate
How do you know if estimates were accurate? Do you track this?	Tend to be accurate 90% of the time	Track changes through changing the estimate in the story Never really sure on accuracy, dependent on who gets work Why it’s important to talk as a group Velocity used as more accurate measure Can use as a tool to figure out ‘why’	Estimate accuracy isn’t too important accuracy isn’t necessarily the end goal Estimates are just a tool to help plan manageable amounts of work.		It doesn’t really matter how accurate they are in terms of which ones are prioritized as long as the difference isn’t super large
Do you or your engineers point stories before hand?	Sprint planning: Everything discussed with tech lead and that was pointed only get moved to sprint Rare to put stories on sprint that haven’t been pointed at this time. Likes having them all pointed before (gives them back to TL)	Yes, have to think of cost Use planning to ‘rope client in,’ spares hyper-technical details Plannings are about the value delivered in the sprint, refinements are for the technical details	Sooner the estimate the better 100% unanimous Point of number is that it’s not time E.g. one team’s 5 could be a week, one’s could be a month Planning Poker has everyone making a call and starting a conversation Don’t want to talk about implementation About how complex the work is, not about how we’ll do the work. ⭐ Planning meetings go long when people discuss implementation Avoid by setting goals for the meeting Having a structured process between teams and projects Good prep — PM review tickets before, and as a group ⭐ Engineers should review the backlog before hand. So they come with good questions.	Will chat with engineers before hand for effort, estimate, No questions during meeting, everything is resolved before Point stories before sprint planning Senior engineer will review and then ask for review	Realistically no, they don’t look at it so it’s necessary to ping the best person for specific tickets Even if devs reviewed before the meetings would still have long discussion Yes?
PT habits	Steve has used Jira more		Prefers Jira		Split between PT and Jira, Trello
How do you set up PT? (Panels/Labels/Icebox/Backlog)	Different sprints, stories can be moved between sprints Backlog is prioritized by the client	Icebox = WIPs, ideas, etc. Backlog = work we can do at any time, not too many dependencies, ‘goals’ System defaults of PT Project history open on the side always Annoyed w/ estimated dates, generally ignore them Use markers for organizing the backlog Backlog/Icebox Backlog is organized by markers	Set up epics track features the client talks about Icebox: ideas, feature thoughts, document and prioritize these Backlog: tickets that are understood and can be worked on and estimated Acceptance criteria Notes Prioritized to show value to the client Current iteration: tickets we committed to during sprint meeting	Uses a “Current Sprint” Hard to setup the next sprint in PT if stories are unable to be accepted due to blockers Velocity is important to setup PT Labels used for what is missing: Needs discussion Needs design Uses ‘Release flags’ as milestones: everything above this flag needs to be done before demo	Release marker in Icebox
Is your whole team in PT?	Yes	Yes		Yes	Yes
How do you set up your stories?	Amica demands this Usual description = As a __ I want __ to be able to do __ Designs are done before sprint starts Copy is the source of truth, not the designs API information Acceptance criteria is for the Amica testers API and test users are what leads the pointing the most for the TL	Title = the story As a….. Details = links to refs, acceptance criteria/specifications/requirement, labels	Stories written from the user’s perspective As a … I would like …		Title → keep it shorter for readability Harder to find In Description → As a…. “whatever the customer wants”
Do you include designs in your stories?	Yes	Yes	Yes
How do you add details? Description vs comments?					If there is too much detail you probs need to break it down Not worried about acceptance criteria
What point scale do you use?	Fibonacci	Dogs Points were noted as arbitrary to some degree		Fibonacci
How do you define what stories get pulled into a sprint planning? How do you define this in PT? Labels/Panels/Backlog?	Sometimes in the middle of a sprint things get moved in Bugs get moved in Anything the client asks for that is critical and wasn’t there before	Icebox is the wild west Moved into the backlog once pointed. Then pulled into a current sprint	“JIRA guy” Customization in JIRA is helpful to standardize things. “Current Iteration” / “Done” / “Backlog” / “Epics” Goal as a PM is to facilitate the work, not to manage people. Rather than swimlanes, Gavin uses “states” in PT States are helpful for QA, and teams members knowing what states to look for Prioritization of the backlog shows value to the client Once a story is pointed, devs with create tasks within the story, to show the breakdown of what a story entails	Keep both the backlog and the icebox prioritized Make sure the stories are correct and pointed Get more details on whatever is next in the icebox Prioritized based on the client’s demos needs
What is working/not working with PT/your project management tool?	Clunky Editing and formatting, ui not elegant, lots of redundancy				Jira: easy to make discussion but workflow is more complicated too many workflows subtasks are hard PT: release marker is annoying but it’s simpler board is harder to read than Jira autoplanning is annoying
Pain Points
What are the biggest frustrations with estimating stories ~~sprint plannings~~?	Not many, they’re going well/smooth and on time ⭐Nice to have: if more devs knew about the stories in advance, the estimate would be more accurate No adjustments during sprint Never a large discrepancy (2 vs 3, not 2 vs 8)	Sometimes details just don’t get read Need feedback as to why Balance between discussion and letting people work Prefers devs do all the estimating Would prefer stories to be estimated well in advance, but doesn’t want to bug the devs ⭐ “Check out the backlog and let me know what you think” — asks all the time. People don’t always do this. In an ideal world devs would be aware of stories before hand. “Why” a story exists is important for the discussion. The goal of a story.	Not having a shared/standard idea of story pointing. Especially for small teams, quick projects. Easy to fall into thinking of points as time. Hours can vary by engineer for the same story, but story points should be the same. Estimation on things they don’t have enough detail for or polished design	Estimation and delivery We don’t do a good on estimating delivery correctly Completion of a story is unclear, the definition of sone is not established from the beginning As long as there are no dependencies on other features, the story is complete	Difficult with a remote team the hard part is thinking through tickets and what is involved. Hardest to get buy in, poke enough holes, Difficult to get people on the same page on what the problem is No frustration on actually pointing the tickets themselves Bias created when discussing every problem in a ticket, causes stagnation After the hard parts, going through requirements the pointing is easy Remote work issues: Time zones Whiteboarding, explaining/sketching ideas really quick can be difficult Body language, interactions in the meeting
What do you see as the cause of inaccurate story pointing?	Data formats etc. in order to use it	Misunderstood reqs and risks	Not well-understood Involves being proactive, having design artifacts Gavin writes stories from a user perspective, so sometimes API or UI work gets baked into those stories. Can make estimating more complex Accuracy isn’t always necessarily the end goal.	Story is not being properly defined: not detailed or is not well understood Dependencies on another API or someone else completing something
~~What is the biggest impediment you see to accurately estimating stories?~~	Edge cases that the team hadn’t thought about. For ex random state policies in Amica
Anything else you’d like to add?	E.g. Issues with MVP estimator, caused by story discrepancies, but never changed size ⭐ Need to be extremely flexible, not all team does it the same way Basic functionality that everyone will need What venue (share w slack, url to send) More necessary than original thought for configuration aspect E.g. who gets final say (majority vs. TL vs. wanting feedback but auth still pointing) ⭐ Way to enforce feedback (Github Reviewers, as an example)	Having the point history is helpful for Lauren to go back and use as a reference. Meat Grinder = tool where requirements risks and estimation go in and well pointed and understood user stories come out ⭐ Make sure techleads or devs actually look at stories and know who has reviewed stories.	Interested in formalizing the process in mojotech is a win. Engineers would know what to expect when jumping on projects Uses slack for “planning poker” Most frustrations are around doing things remotely. Supporting remote teams would be key for Gavin Leverage existing tools rather than adding a new one. Once stories are in the current iteration and pointed, engineering team adds development details Gavin’s PT proces: Link		Getting people to review tickets ahead of time would make meetings go a lot faster having informed discussions is a huge issue PMs sometimes write out tickets last minute too Remote meeting challenges effect sprint plannings just as much
Follow up:	Chat with Joe A. about how he points, what he needs to estimate, how he gets feedback… etc.	Ask Lauren more details about what point scale she uses. She’s used planning poker in the past, but found people get stuck picking a card (usually between 1/2 or 2/3). Usually starts by asking someone to start the pointing, and then go around the room and ask "Do you agree with so and so?" She keeps it a group discussion, but sometimes might ask someone to point something in Slack (like my tech lead can point it ahead of time), and then will confirm with everybody in the estimation meetings. Ask if she does “point poker” style When using dog sizes Lauren will later translate to Fibonacci.

PM Interviews - Summary

	Steve	Lauren	Gavin	Furhana	Paul
Engineers reviewing stories before hand	Pointing Stories On Amica, the tech lead (Joe) points the stories before the refinement, and will ask for a second opinion from other devs if necessary. This has proved efficient and accurate. Uses Fibonacci Creates a JIRA board for the upcoming sprint, adds stories to board before sprint planning. If stories points change, usually won’t change the original estimate. (Because it’s often only off by 1 point, or cause the extra work is broken out into a separate story) Frustrations If more devs knew about the stories in advance, the estimates in meetings would be more accurate. Estimates are most inaccurate when there is unknown data, creating extra use cases, more work to work with the API, etc. Ideas It could be nice if there was a way to enforce (or confirm) that the tech lead’s story estimate was reviewed by a team member(s). (such as GitHub reviews) Any tool will need to be flexible to fit different PMs styles. Would like story links to be shareable in slack. (or at least the ability to copy a link)	Pointing Stories Points stories as a team. Usually starts by asking someone to start the pointing, and then go around the room and ask "Do you agree with so and so?" Often the engineer who is assigned the story will have final say. Sometimes asks the tech lead to point it ahead of time, and confirm with the team in the estimation meeting. Tries to abstract points in meetings into tshirt sizes or dog sizes, but documents points using Fibonacci. Ask devs to look at stories before the refinement If the points aren’t right, then change them, and add as comment in PT with original point, and reasons. Having the point history is helpful for Lauren to go back and use as a reference. Finds that the goal/“Why” of a story is important for the discussion, as well as technical details. Frustrations Would prefer stories to be estimated well in advance (to the refinements/planning), but doesn’t want to bug the devs Often finds herself asking “Check out the backlog and let me know what you think” — asks all the time. Engineers don’t always do this. In an ideal world devs would be aware of stories before hand. Would find it helfpul if she could make sure tech leads or devs actually look at stories and if she knew who has reviewed stories. Inaccurate estimates come from misunderstood reqs and risks	Pointing Stories Uses slack for “planning poker” Tries to estimate stories as early in the process as possible. Wants 100% unanimous team agreement on point value Likes Planning Poker cause it has everyone making a call and starting a conversation Gavin thinks tools like planning poker are important to allow conversation and differing opinions. — in order to gain consensus. Frustrations Most frustrations are around doing things remotely. Not having a shared/standard idea of story pointing. Especially for small teams, quick projects. Easy to fall into thinking of points as time. Inaccurate estimates come from not well-understood stories Involves being proactive, having design artifacts Planning meetings go long when people discuss implementation Avoids this by setting goals for the meeting Having a structured process between teams and projects Good prep — PM review tickets before, and as a group Engineers should review the backlog before hand. So they come with good questions. Ideas Interested in formalizing the process in mojotech is a win. So engineers would know what to expect when jumping on projects	Pointing Stories Talk offline about the estimate, then confirmed during the spring meeting Compared to previous tickets and estimates Senior engineers point stories accurately since they have more experience Small projects: Pointing Poker in person, but people gravitate towards the most senior person Larger projects: tech lead does the story pointing and asks for feedback Will chat with engineers before hand for effort, estimate, No questions during meeting, everything is resolved before Point stories before sprint planning Senior engineer will review and then ask for review Frustrations We don’t do a good on estimating delivery correctly Completion of a story is unclear, the definition of sone is not established from the beginning Ideas	Pointing Stories Sometimes play poker, other times just ask for suggestions (go one by one) Just picks a point value if people are evenly split (2 vs 3) Frustrations Move tickets that feel ready to review before planning meeting, so people can read them before hand, but they rarely do Realistically, engineers don’t look at stories beforehand so it’s necessary to ping the best person for specific tickets Even if devs reviewed before the meetings would still have long discussion Difficult to get people on the same page on what the problem is No frustration on actually pointing the tickets themselves Remote work issues: Time zones Whiteboarding, explaining/sketching ideas really quick can be difficult Body language, interactions in the meeting Ideas

Dev Interviews

What is the usual lifecycle of a story on the teams you’ve been a part of?

What do you consider are the most important factors in a story?

How do you estimate story points with your teams?

What have you found to be the most effective way to do this?

Does it change project to project? Why?

Have you done it differently in the past?

What criteria do you consider when estimating a story?

What criteria do the rest of the devs in your team consider when estimating a story? Does it match yours?

Are there any recurring discussion points between the developers when there is disagreement in story estimates?

Does your team have a preferred scale for estimating stories?

Do you think pointing stories is important? Why?

Do ticket estimates change how you approach a story?

Are your estimates usually accurate?

How important is it for you that ticket pointing matches the actual work?

What do you think are the usual causes of inaccurate story pointing?

What are some ways you consider would help developers to better estimate stories?

Are there any things you would change about the way your team currently does story pointing?

Are there any frustrations with refinement/sprint planning process?

Do you use the voting/poker style of pointing? Why/why not?

Anything else you would like to add?

	Joe A.	Matt F. (2:00)	Seth (3:00)
What is the usual lifecycle of a story on the teams you’ve been a part of?		Start after conversation between client and PM Division into stories by PM and show to team lead to make sure it’s estimatable: valid → estimatable Before planning, determine acceptance criteria for dev and non-devs “The site is too slow” is an example of an unestimatable story Pointing has to precede prioritizing because estimate leads that Prioritize on a schedule	Refinement = fact checking Might speak to person who might be assigned feature to estimate
What do you consider are the most important factors in a story?			Clear DClear definition Actionable TODOs to get a sense of progress Actionable todos
How do you estimate story points with your teams?		Pointing poker in a small team Valid stories that have been shown to the client → read them out loud (devs look at them before) → 3,2,1 on Slack Depending on the discrepancies they discuss or not Pointing Poker gets more useful the larger the team, and is more useful when there is low trust between the engineers and clients	Often ties point to days 1 point is roughly a day Most efficient for PM & Tech Lead to groom and roughly estimate the story before planning Then have the rest of the devs look at the stories in a more official planning meeting
What have you found to be the most effective way to do this? (do you have a preference)	As the tech lead, Joe does most of the story refinement, so easy to point from there Pointing Scale 1 - 30sec-2hr 2- 2hr-day 3 - day-2days 5- 2-4 days will ask for second option on most stories (besides 1 pointers) —	Benefits of poker style Any one persons estimate is going to be bad - gives everyone a chance to fail Allows for failsafes Fast, puts people on the spot Estimating separate isn’t always the more efficient use of time. Disadvantages of poker style It’s not necessary after a long time working with a specific team It only works when working with a new team/client/project Team does not usually do it, but prefers this method
Does it change project to project? Why?
Have you done it differently in the past?
What criteria do you consider when estimating a story?		Time, complexity Strong opinion on points = time, because client does not care about things like complexity for ex Unit that client can understand completely Disagreement on what the value of a point is
What criteria do the rest of the devs in your team consider when estimating a story? Does it match yours?
Are there any recurring discussion points between the developers when there is disagreement in story estimates?			Often disagreements Individual devs opinions are valuable cause they might have more specific domain knowledge in specific features Their vote counts more than others
Does your team have a preferred scale for estimating stories?			Fibonacci
Do you think pointing stories is important? Why?			Personally thinks it helps Some clients don’t value points at all
Do ticket estimates change how you approach a story?			Gives a frame of mind on how long it will take draws out a roadmap
Are your estimates usually accurate?			Finds estimates to be mostly accurate
How important is it for you that ticket pointing matches the actual work?			Not really super important that tickets are accurate, though they do help with statistics Doesn’t really effect sprints all that much, though it might change developer focus
What do you think are the usual causes of inaccurate story pointing?			Unknowns Lack of knowledge into the domain of a feature
What are some ways you consider would help developers to better estimate stories?			Insuring that the Pm creates a business level plan knowing what the business values in a ticket informs the developer on decisions Identifying known unknowns beforehand
Are there any things you would change about the way your team currently does story pointing?	thinks bad idea to point during sprint preplanning — cause no one has an opinion in the meeting	⭐ It’s not that bad but it’s unofficial and there is no way to store the information for later ⭐ Discussions on the different estimates are not recorded anywhere When pointing isn’t accurate there is no collection of what they did wrong other than just a discussion in retro No way to compare discussion of estimates with the stories--- can’t go back and look at the discussion history ⭐ its’ also fun ⭐ thinks standardizing a process would be good for mojotech	Tangents, people talk too much about unrelated content discussion become unanchored; a theoretical discussion instead of the topic at hand this typically happens when there a re a lot of unknowns Limit time with a timer, if we go over then story needs to be redefined
Are there any frustrations with refinement/sprint planning process?	biggest frustration is getting client feedback in order to properly refine a story
Do you use the voting/poker style of pointing? Why/why not?	No poker not on amica not on any project in the past Doesn’t like poker, cause the person who refined has the most knowledge about the story	Yes for most projects, it works, it’s fun Not currently because it’s a small team and they have worked together in the past Maybe once a month, not totally gone but it’s not usually necessary	Hasn’t used poker style Tried it once, issues w scope creep, not getting enough feedback from client
Anything else you would like to add?		Standardizing the approach would be beneficial for MojoTech

Review

How did these conversations challenge our assumptions?

What do we offer of value to the people we spoke with?

What are the greatest challenges we face getting them to adopt our product?

How else might they already be solving the problem we want to solve?

What are the greatest opportunities to serve them?

Product Definition

How might we…

Facilitate informed and efficient refinements & story estimation

Facilitate better discussions for remote teams

Create a better system for poker voting in meetings—requires remote people to share, opens the door for discussion when they disagree

Encourage all parties to gather all necessary information for a story before the refinement

Encourage teams to review stories before refinements when needed

Allow PMs to tag engineers in specific tickets for review

Allow engineers to see a curated list of tickets that need their review

Require voting on a story before hand?

Encourage a shared understanding of point value

Integrate with existing PM & tech lead workflows

Use labels or release markers to automatically pull stories into the tool?

Push comments and the final story estimate into the story in PT once decided?

Simplify and streamline story estimation within a refinement

Better UI designed for story pointing and discussions

Create a way to poker vote on estimates

Document the discussion around story estimates

Allow PM to add discussion notes that get added as a comment in PT

Mariana Pachon-Puentes 6 years ago (edited)

I think maybe we should start with a general PT or story contribution question s

Chapman Bettis 6 years ago

Yeah, good call. I think the “Lifecyle of a story” question is a good open ended

Mariana Pachon-Puentes 6 years ago (edited)

Not sure how to ask this without it being a leading question, but just trying to

Chapman Bettis 6 years ago

How about “Do you find pointing import to your development work?” or “Does the e

PT Estimator
Links
Similar Tools
Slack tools
PM Interviews (Full Notes)
PM Interviews - Summary
Dev Interviews
Review
Product Definition
How might we…

Give this doc a home

​​Links

​​Similar Tools

​​PM Interviews (Full Notes)

​​PM Interviews - Summary

​​Dev Interviews

​​

​​Review

​​Product Definition

​​How might we…