Re: [tor-dev] Survey on Tor Trac usage and how you manage your tasks

[Karsten Loesing <karsten.loesing@xxxxxxx>]

On 8/29/11 10:48 AM, Karsten Loesing wrote:
> The new deadline is
> 
>   Sunday, September 4, 2011
> 
> Dear Tor developers and volunteers who haven't responded yet: If you
> have an opinion on using Trac, please let us know!

First of all, thanks to the 13 people for taking part in the survey!

In this mail I'm trying to summarize the results from part 1 of the
survey and derive some suggestions for making Trac better.  (I'm leaving
out the results from part 2 for now, because it turned out the questions
weren't as useful as I thought.  I may use the replies to design a
second survey in a month or two from now, but for the moment I think
it's best to just focus on our Trac usage.)

Below are the results, plus some suggestions in the paragraphs starting
with "[Suggestion: " to separate my suggestions from result summaries.

Feel free to comment on the suggestions or add your own!

Depending on whether this thread is going to explode, I'll write another
summary in a few days to list the suggestions that people seem to agree
on and that we're going to implement.

Best,
Karsten


> 1 Using Trac features
>
> 1.1 Which of the reports (stored ticket queries) do you use most often?

From all the replies, there's 1 person using 5 reports, 2 persons using
3 reports, 1 person using 2 reports, 4 persons using 1 report, and 5
persons using no reports at all.  There's only 1 report being viewed by
more than one person.  5 persons are not using reports at all.

The following reports are viewed: 7, 8, 12 (mentioned twice), 14, 22, 23
(mentioned twice), 27, 28, 34, 35, 36, 38, 39, 40 (mentioned twice).
Hence, the following reports are not viewed: 1, 2, 3, 4, 5, 6, 10, 11,
15, 16, 17, 18, 19, 20, 21, 24, 29, 30, 31, 32, 33, 37.

[Suggestion: Backup and delete all reports with a component name in
them.  The current list of Available Reports list is mostly useless for
newcomers who don't care much about components, but who are interested
in finding something to work on.  Developers and volunteers can bookmark
custom queries or put links to them on a wiki page belonging to a
component.  For example, report 12 "Tor: Active Tickets by Milestone" is
https://trac.torproject.org/projects/tor/query?status=!closed&group=milestone&component=Tor+Relay&component=Tor+Client&component=Tor+Bridge&component=Tor+Hidden+Services&component=Tor+bundles%2Finstallation&component=Tor+Directory+Authority&order=priority&col=id&col=summary&col=component&col=status&col=type&col=priority&col=milestone&col=version&col=keywords
]

> 1.2 What are typical custom queries that you run?

There were 11 replies to this question.  4 persons start from a small
set of queries that are always the same, similar to stored ticket
queries, and refine them as needed.  4 persons run many different
queries.  4 persons don't use the custom query feature at all.

> 1.3 How do you use milestones and roadmaps?

7 out of 11 people use milestones.  The components for which milestones
are used are Tor (4 mentions), Vidalia (1 mention), TBB (1 mention), and
sponsors deadlines (1 mention).  4 persons use wiki pages or parent
tickets instead of or in addition to milestones to classify when tickets
should be completed.  Nobody uses the roadmap view of milestones.

Unrelated to this survey, here are some statistics on the milestones in
our Trac:  We have 224 milestones in total, 37 of which having at least
1 ticket assigned.  That leaves 187 milestones with zero tickets
assigned, most of which are a left-over from the flyspray migration.

[Suggestion: Delete all milestones that have no tickets assigned to them
or that seem useful even without assigned tickets.]

> 1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do
> you use the mails sent to these lists?

11 persons are subscribed to tor-bugs, but most people filter the mails
to only learn about components they care about.  5 persons read
tor-wiki-changes, but at least 3 persons would be glad if they contained
diffs.

[Suggestion: Try harder to include diffs in tor-wiki-changes
notifications.  Maybe Trac 0.12 will make this easier.]

> 1.5 Which wiki pages do you read/edit most often?

8 out of 11 persons replying to this question use the wiki.  Mentioned
wiki pages are (multiple selections allowed): sponsor pages (5
mentions), specific pages in doc/ (3 mentions), various FAQ (2
mentions), dev meeting pages (2 mentions), roadmaps in org/roadmaps (1
mention).

> 1.6 How do you search for wiki pages?

People use at least four methods to search the wiki (multiple selections
allowed): TitleIndex (5 mentions), external search engine (4 mentions),
browser auto-completion or history (3 mentions), Trac search (3 mentions).

> 1.7 What are typical search terms that you use when using the search
> features?

3 persons search Trac using key words and 1 person types in ticket
numbers in the search field.  The rest doesn't use the search feature.

> 1.8 Do you use keywords and the tags page, and if so, what are keywords
> that you typically use?

1 person uses Agile iteration tracking keywords (D'oh, there goes the
anonymity!) and 2 persons use the "easy" keyword sometimes.  At least 1
person was surprised that there's a tag page.  The rest doesn't use
keywords, nor the tags page.

> 1.9 How relevant are the following ticket fields for you?

Here are the ticket fields sorted by conceived relevance in decreasing
order, plus some comments.  The numbers in brackets are (useful,
somewhat useful, not useful):

The Component field (10, 1, 1) is used to find/filter tickets and guess
who's paying attention to a ticket.  1 person said that the many Tor
components make it hard to refer to the software "tor" and that it's
easier to look at milestones itself.

The Owned by field (10, 1, 1) is important for most people to decide
whether they're responsible for fixing or finding someone else to fix
something.

The Milestone field (9, 0, 2) is used for release planning and sponsor
deliverables, and it also determines the ticket priority to some extent.

The Reported by field (8, 2, 2) is used by some to estimate how useful a
reported defect or suggested enhancement is, and it is useful to know
whether someone will be mailed by Trac when the ticket changes.

The Cc field (7, 1, 3) is used to add people whose input is desired and
to learn who will be mailed when the ticket changes.  There's the
problem that only users with certain permissions can add other persons
to the Cc field of existing tickets.

[Suggestion: Allow all developers and volunteers to edit the Cc field.]

The Priority field (6, 5, 1) is important to some people, but is highly
dependent on who sets the priority.

The Parent ID field (6, 1, 4) is used for complex tickets with many
subtasks and when a task spans multiple components.

The Version field (3, 3, 5) is not used by many components and is
considered not very useful, because bug reporters get versions wrong in
most cases anyway.  Also, current versions of products are never in the
list.

[Suggestion: Delete all obsolete versions from the list, and try harder
to add new versions.]

The Keywords field (2, 1, 8) is only used for agile iteration tracking
and to tag tickets as "easy."

The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively
used by only 1 person and read by 1 other person.

[Suggestion: Investigate whether it's possible to suppress email
notifications for changes to the Points and Actual Points fields.]

> 1.10 How relevant are the following ticket statuses for you?

Here are the ticket statuses sorted by relevance from most important to
least important.  Numbers in brackets are (useful, somewhat useful, not
useful):

The closed status (8, 0, 2) is considered useful by pretty much everyone.

The needs_review status (7, 2, 1) is used to determine which tickets to
read and respond to first.

The needs_information status (6, 2, 0) is new, but considered promising
and says that one cannot do more without feedback.

The new (2, 1, 4), assigned (1, 3, 5), reopened (1, 2, 5), and accepted
status (1, 1, 6) have slightly different meanings for some people, but
are mostly equivalent for the majority.  There are 5 persons who'd like
to see these four statuses merged into one, e.g., assigned.  1 person
suggests to narrow down statuses to three statuses: assigned, pending
(with a combo box for reasons: requester information, code review, code
push, schedule), and resolved.

[Suggestion: Investigate whether we can merge "new," "assigned,"
"reopened," and "accepted" into a single "assigned" status.]

> 1.11 What other features do you use in Trac?

4 out of 11 persons mention use of extended features.  These features
include admin functions, e.g., deleting users and spam tickets, embedded
queries (see #3410), and RSS feeds for custom queries, tickets, wiki
pages, and the timeline (2 mentions).

> 1.12 What features are you missing in Trac?

Better search: The current search is ridiculous. I need to specify
things like "I want these three keywords definitely, only in torbutton
tickets that are in status needs-review".

Git commit hooks (2 mentions): We should be able to mention "Bug #XXX"
at the start of a commit message and have that commit get posted as a
comment to trac w/ a link to the gitweb url for the commit. I believe we
have a trac plugin for this, but it is not yet properly configured. At
other employers, commits could only go into stable releases if they
referenced a bug # in the first line. I thought this was a good policy.
Back in the SVN days, this property was actually enforcable on the
server. Git may make this bit more difficult..

[Suggestion: Investigate Git integration in Trac 0.12.]

Trac 0.12-stable: I would like some of the embedded query syntax that is
available in 0.12.x-stable of trac. In particular, 0.12 would make it
much easier to compute the rate of Opened tickets against a given
project in a certain period of time.

[Suggestion: Try out Trac 0.12 on a VM and find out.]

Better handling of duplicate bugs: The "dup" feature should have a bug
field that causes both bugs to be updated automatically with links to
each other.

Field updates should perhaps not send email: Some field updates don't
require emailing everybody on the ticket. I *think* 0.12 might provide
the ability to solve this one, too, but I am not sure.

Prevent trac from emailing you about your own changes (2 mentions):
always_notify_updater = false in trac.ini is close to this, but it is
not exactly what I want. I think it is useful to automatically notify
previous updaters via email, but it seems silly to send me mail for my
own changes. Yet there is only one option for both behaviors in trac.. I
could see setting it to false and just being more careful about adding
people to the Cc line as a stopgap.

Embargoed/Security tickets: For serious security issues, it would be
nice to be able to make tickets not viewable by accounts not on the Cc,
owner, or reporter lines of the ticket. I don't think trac can do this
at all.

Component-specific email notifications: I would have preferred the
ability to receive emails for any changes related to a specific
component (i.e. TorCtl).

[Suggestion: Find out if r2e can do the trick and explain this on the
wiki somewhere.]

Better anonymous reporting (3 mentions): (First reporter) I'd like
better anonymous reporting and commenting.  (Second reporter) I've
bolded the 'New Ticket' and cypherpunks info on the landing page, though
creating a ticket via cypherpunks really should be a fist sized cherry
red button on that page...  (Third reporter) I think the 'cypherpunk'
user should have to add an email address in the Cc field. If they put
mailinator in there, so be it. But they at least need to know we need to
talk to them, or their bugs will probably just sit around forever.

Voting on issues.

Tor-detection: It would be nice to have Tor detection built into tor -
so we can see if a reporting user is actually using Tor!

Multiple milestones: I wish I could assign bugs to multiple milestones.

[Suggestion: Only assign defects and enhancements to software product
milestones and only assign tasks and projects to sponsor milestones.]

Large-scale ticket management: One of the main problems we have in
general is the proliferation of tickets. Originally that was a good
thing -- "have something we need to do? Make a ticket, then we'll
remember." But now we have so many darn tickets, especially in some
components, that they're tough to manage, which leads to less active
maintenance, which leads to having it get even more out of control. I'm
not sure what to do about it, but sometimes it makes me less excited to
add more tickets since I feel like I'm adding to the problem. So some
feature to do large-scale ticket management would be good.

> 1.13 What features would you want our Trac not to offer anymore, because
> they're making things only more confusing for you (and for people who
> are new to Tor)?

No email notifications for certain fields: points, actual points are
annoying because changes in them cause email to be sent as an update. cc
list is the same.

Separate system for users and developers: Trac is used for user support
too much. I wish we had a system that was JUST developer centric, no
user wiki pages or other crap that turns up in searches.

Landing page: Our landing page is a nightmare for newcomers.

[Suggestion: Write a new landing page.]

Single "tor" component: I don't know what confuses others, but IMO the
proliferation of components that are all "tor" doesn't help me, and
makes stuff slightly harder.

Get rid of wiki: Most of the contents of the wiki consist of obsolete
and/or bad advice.  These pages should be archived in a tarball where
they will not be readily accessible to web browsers and search engines,
then purged from our Trac installation.  The rest of the pages in the
Trac wiki should be moved to a wiki which allows a user who edit a page
concurrently with another user to merge his/her/its changes into the
wiki page.  Perhaps we should just use Git and give up on browser-based
wikis.
_______________________________________________
tor-dev mailing list
tor-dev@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev