What makes a good mentored bug?

Mentored bugs are basically the same as good first bugs, but with a mentor. That means we can use the same criteria as before:

• Easy:

  The best possible first bug, from a procedural point of view, would be a typo fix [3]. It would be trivial to fix, but would serve to teach you bugzilla, build firefox, meet other developers, format your patch correctly, ask for review, and so on. These procedural issues are what makes contribution difficult, and a perfect first bug would just guide people through these steps.

• Self-contained:

  Bugs which depend on other bugs, or which span many modules, are probably not great first bugs. The contributor will be unused to Mozilla’s practices, and having to split work across multiple trees, figure out super-review, etc, are not going to encourage them to stay.

• Well-understood:

  You should be able to guide the contributor down the right track, which means you should understand the bug yourself. Similarly, the bug should have a good start and finish – it should be obvious when the bug is fixed. Bugs which contain a reduced test case are perfect for this; intermittent oranges are not. In general, you should know how you’d approach it, and you should add that as a comment: how to reproduce the problem, the file and function to fix, how you’ll recognize the required fix, etc.

• Apolitical:

  The last thing you want is for the contributor to pour hours into fixing a bug, only for it to be nixed by the reviewer or module owner. You should be 100% certain that fixing the bug is desired, and that there won’t be any dissent at a later date. We don’t want newcomers leaving because their time was wasted.

Although we say “easy” above, we can also mentor bugs that would serve as good second and third bugs. Second and third bugs can add a little more difficultly and discovery. They should still be conceptually easy, but hold a small challenge for someone who doesn’t know the codebase. They’ve jumped through the hoops, now reel them in with something slightly meaty and interesting [4].

What kinds of bugs are useful to mentor?

In general, they should be bugs you’d like to see fixed, but ones that aren’t so important that you’ll do them yourself. Good examples are:

• Small bugs you don’t have time for:

  For example, the JS team wants to integrate our two testing frameworks, but we probably won’t do it ourselves.

• Bugs you’ve started, but won’t finish:

  I’ve started a few bugs that I ran out of time to finish, but I still want them to get done. In those cases, I attached what I had done and marked them as mentored. These bugs are particularly good since they are really close to being finished, and the trajectory a new contributor needs to take is unambiguous.

• Broken windows:

  Mozilla is an old project, and has accrued many broken windows – things which are an incredibly low priority to fix, but make the place look ugly and run-down. These are ideal starting points, especially if they only involved deleting code. Beware that this can lead to scope creep: a new contributor recently removed WinCE support from SpiderMonkey, which had to be split into 5 separate bugs requiring 5 separate reviewers. This can scare new contributors off [5].

• Refactorings:

  Refactorings are a very good way to learn a code-base, though they won’t always be good first bugs (perhaps second or third bugs). It’s always great when a contributor can add something of beauty, instead of a hacky fix.

• Local optimizations:

  If you have a single function that could be optimized, that makes for a very self-contained bug. The new contributor can write a micro-benchmark and optimize it, and really see the results of their work.

Next steps

We’re currently directing newcomers to an introductory page, where we link to mentored bugs, and encourage newcomers to pick a bug they’d like to work on. It seems that giving newcomers bugs directly is far more effective than leaving them to find one they like, so we’d like to move to that if possible. Unfortunately, we don’t have enough mentored bugs, or a wide enough variety of them, to make this work just yet.

This is particularly true of non-C++ bugs. We have lots of newcomers who don’t know C++, or who barely know it. We currently have an introductory page for people who don’t know C++, but we’d like to do a lot better here. If you work on a aspect of Mozilla that doesn’t use C++, whether it’s RelEng, a website, or even a small tool, we’d love feedback on how to help newcomers work with you.

Speaking of feedback, we want lots of it. Mentored bugs are new to Mozilla, though they’ve been used in other projects before. We’d love to hear your experiences with them, here and elsewhere, to see if we’re really improving things. Our contact details are in the first post, and we’ll keep the mentors page updated as we go.

[1]	We’re in early stages here, so feedback welcome.

[2]

We don’t want new contributors to be frustrated by having to wait days or weeks for a review. “John here is a new contributor, would you be able to prioritize this review?” works wonders; even if it doesn’t speed up the review, it makes the contributor feel special. I’ll get into this more in a later installment.

[3]	However, since it wouldn’t give the contributor a warm fuzzy feeling, probably best not to file these.

[4]	For example, here’s a bug I wrote specifically for a new contributor. It’s a challenging bug, but is self-contained and well-understood.

[5]	Though it didn’t in this case, cause he is awesome. He actually then went on to delete WinCE from the whole damn tree!

Challenges for contributors

• What does a new contributor have to go through to build FF:
  • no readme
  • —enable-application=X
    • no warning
    • needs default
  • directory is called mozilla-2.0, not firefox4.0
  • cvs is mentioned in the docs!
  • build instructions prose is poor
  • not configure/make based
  • ac_add_option / .mozconfig is confusing
  • patch submission is tortuous
  • even simple stuff has jargon
• Getting feedback:
  • there isn’t a good wasy to ask “what do you think of this approach?”
    • f? only works on attachments
    • some people put the proposal as an attachement, then ask for feedback
    • there was a suggestion of a ‘request review’ mailing list.
• Tinderbox is scary:
  • new contributors are scared by oranges, and we currently have a problem with random oranges
    • there is automated oranging, sufficient?
    • ArbPL might be easier in this regard
• Submitting bugs:
  • finding a committer is tough
    • we should have a script/query/dashboard following [commit-needed]
    • add a “job” like sherriffing, for the person whose job it is to land patches
  • getting tryserver access takes a day
    • arbitrary try server access
      • solvable, but security concerns
  • we’d like to prioritize landing from new contributors and “people who aren’t paid to work on this thing”
  • currently work has to be ‘negotiated’ into the tree
    • new contributors should have ‘greased path’ into the tree
• So many tools to learn:
  • tbpl
  • tryserver
  • qimportbz
  • bzexport
  • gdb-archer
  • lithium
  • irc
  • mercurial queues
  • bugzilla fastness
  • weird unique build system – autoconf213, mozconfig
• Time zones:
  • hard to get an answer if you contribute outside 9-5, Monday-Friday PST
• Working independently: Problems should be solvable for external contributors independenctly. Each new person who is introduced can potentially scuttle what you’re doing. If you email someone and wait a week for a response, it makes it less fun. If they never respond you’ll probably stop contributing. Related: tryserver access, reviews, approvals, commit-needed

Experience reports

• Long form:
  • Steve Fink
  • Josh Matthews
  • New to Spidermonkey
  • Cameron Kaiser
  • TabCandy
  • Joshua Cranmer
• briefly:
  • seamonkey does a lot of hand-holding, and has been successful
  • l10n too
  • Gnome Love project
    • (gnome-women is also copyable)
  • maybe get reports from professors, Seneca, etc
    • relationships are more important than resources
    • you need to “protect” new contributors from “normal” interactions
• Experiences summary:
  • idiosyncratic build infrastructure
  • submission process (including bugzilla)
  • contributor agreement
  • r? with no target are ignored
  • requesting reviews
  • learn irc, our channels and etiquette
  • blame logs
  • finding related bugs in bugzilla
  • [checkin-needed]
  • project branches
  • watching the tree + tinderbox
  • approvals process
  • blocking/wanted flags
  • forgetting the -u flag [editor’s note: NEVER EVER FORGET THE -u FLAG!!!]

Engagement

• Contributor engagement team (ref David Boswell)
  • high level, need coding-specific help
  • 100 people contact per week to help
    • mozilla.org/contribute -> contribute@mozilla.org list [needs volunteers to answer the queries - contact david@mozilla.org]
    • contribute@ list and canned responses
    • poor success ratio (10% is a good result)
    • getting started information
    • #mozillians channel
    • track what works and doesnt
    • point them to first bugs
    • contributor directory can help
• Mentoring:
  • we should have an outreach team
  • mentors would guiding patches through the process
  • danger: watch out for people not going the distance
  • people who learn processes will be good in the long run
• What do we do when potential contributors email existing contributors directly?
  • mentor?
  • send canned response?
  • redirect to Constributor Engagement
• terminology is important:
  • “internal” vs “external” – “internal” might have bad connotations
  • volunteers is bad
  • “greased”?

Tools to help us

• we need dashboards to find:
  • unspecified r?
  • forgotten [checkin-needed]
  • patches from new contributors
  • [good first bugs]
• identify new contributor (metrics):
  • age of bugzilla account
  • number of patches submitted by this submitter
  • whether a submitter is employed by Mozilla Corp
  • whether a submitter is paid to work on Mozilla
  • age of submitter’s first-ever patch
  • timezone of submitter
  • native language of submitter
  • age of submitter
  • number of bugzilla comments written by submitter
  • whether a submitter has commit privs
  • fuzzy heuristic combinations of the above to bucket people into “new unpaid volunteer” or whatever
  • dmose/metrics have built something like this
• development appliance?
• documentation:
  • live help?
  • feedback button (there is a talk page – is that the same thing>)
• a stackoverflow instance like infomonkey for mozilla questions

Documentation

• Is this just a documentation problem? [editor’s note: No]
• many docs are obselete
  • the important ones are the ones which appear high on google
  • hard to update
  • updating guidelines