Initial survey info

[Roger Dingledine <arma@mit.edu>]

Welcome to seul-research. Our current goal right now is to figure out
what we have to do to write a good survey, and then actually write it.

Table of contents of this message:
* People on the seul-research list
* Motivation
* Some initial thoughts on format

After that, I will follow it with a message that is a compilation of email
conversations I've had on this subject so far. (I separate the messages
to try not to flood people with information.)

This message is kind of long. You might want to save it or something to
reference it. seul-research is being archived on the seul site -- eventually
I'll get around to making the seul-research list archive public and that
will take care of that. Comments and replies appreciated. Let's get some
discussion going here. :)

****************************************************************************

First of all, some basic introductions of who's on this list right now:

arma@seul.org -- me. I can write really well, but I don't have any
  prior experience with surveys, and I'm trying to do a lot of other
  seul stuff concurrently. :( I'm one of the two current seul leaders.

omega@seul.org -- the other seul leader. He's just here because he likes
  reading email.

luka@seul.org -- another seul person. Hopefully he'll have time to read
  this list and put in some useful comments.

solaris@seul.org -- the seul-seg (http://www.seul.org/seg/) leader.
  He's done some work on a previous survey (?) in Spanish, and hopefully
  will be able to help out with writing but especially with actually getting
  the survey to people to take it.

swaldman@seul.org -- our webmaster, and one of the seul-pub
  (http://www.seul.org/pub/) leaders. He intends to look at our work and
  tell us when we're being stupid or ambiguous.

hvirtane@cc.jyu.fi
  A volunteer who wants to help write the survey. (Elaboration? :)

pete_st_onge@iname.com
  A volunteer who wants to help write the survey. (Elaboration? :)

brians@mit.edu
  A volunteer (local to me) whom I've convinced wants to help write the
  survey. He's a mac interface weenie, so presumably he knows something
  about end-users.
  
kmself@ix.netcom.com
  Author of the SAS survey
  (http://pw1.netcom.com/~kmself/SAS/sas_user_survey.html) and somebody who
  is clueful about writing surveys. He'll going to tell us when we're being
  stupid.

People who aren't on this list yet but should be:
http://www.linux.or.jp/~pjotr/mc-doc/results.html
http://aachen.heimat.de/alug/fragebogen2/
I've contacted the authors of both these surveys, and have yet to hear
from them.

****************************************************************************

Motivation: why I want to get this survey written.

I want to figure out what end-users want in an operating system, and I
want to let everybody else know.

As quoted from the seul manifesto (http://www.seul.org/what/manifesto.html):
>The end-user (as we use the term) is a person who just plain uses Linux, and
>the applications and accessories produced for it. His main goal is not to use
>Linux for the purpose of creating more Linux, whether for himself or the
>public (e.g. hacking around in his config files and scripts, or writing new
>Linux applications for redistribution, respectively). Thus he is at the "end"
>of the production line. He wants compiled code, in the form of an intuitive
>graphical icon on his screen, which he can click on and correctly expect it
>to do what it seems like it should.

But it goes beyond this, for us -- the end-user we are targetting for this
survey is in general not yet using Linux, and probably doesn't think Linux is
 sufficient for his needs (and he's probably right at this stage). We need to
figure out what our target users are doing right now, and which parts of that
are most important to them. 

This can be put to several uses:
* Developers need a goal: if we can figure out which applications or
  environments are most important to potential new Linux users, then the
  developers can focus on those aspects.
* Advocacy people need a goal: if our survey finds that feature foo is
  very important to our target users and it's already implemented, then we
  can tell the Linux advocates to say "Linux has foo!" more often. While
  the Linux advocacy effort has a lot of zealous members right now, it's
  very disorganized and most advocates aren't as clueful as they could be
  about what will actually sell Linux. I want to have something concrete to
  provide to the Linux community to help education and understanding.

And of course, having a good survey distributed widely will raise the
awareness of end-user issues for Linux, both in the Linux community and
maybe even outside it if it gets distributed widely enough (it's a nice
thought).

****************************************************************************

Some initial very basic thoughts on format:

The first part of the survey will be gathering information on the person,
so we know what sort of person is answering the survey. I suppose we should
ask such things as profession, age?, and experience with different operating
systems. This will let us know what sort of target user is answering the
survey -- somebody who knows about Linux and has used it, or somebody who
hasn't ever touched it before. I suspect we'll divide statistical results
into section based on these initial questions.

Then I want to know what they *do* with their computer. Do you do
word processing? Do they play games? Do they design circuitboards?
Are the applications on their current OS sufficient for each of these?

However, for the "word processor" I would think to myself "yes, I use latex,
and it does everything I need." Given that, I should have some chance to
say "But there aren't any good intuitive wysiwyg word processors for Linux."

We need to find the right compromise between too general and too specific.

Something else to consider is learning what hardware the person is using,
and whether it's sufficient for each of the above activities. 

I guess we could have a separate section for each OS, or we could let the
user fill out a certain section multiple times once for each OS.

We also need to figure out how to word questions about the OS features
themselves -- most people aren't going to know or care if it's virtual
memory or not, but they will care if it can handle multitasking correctly.
But we need to be sure not to bias our questions towards Linux -- if we do
that, we might as well be surveying people with "Do you prefer an OS that
works?"

I want to take a very quantitative approach to the survey: I want a lot
of questions that people can answer without writing words. This makes it
easier to fill out the survey, and it also makes it easier to provide
statistics about it.

--Roger