Re: SEUL: draft survey: v.0.001, just some thoughts

[Roger Dingledine <arma@mit.edu>]

In message <Pine.SOL.3.96.980702155326.13578A-100000@kanto.cc.jyu.fi>, hvirtane@cc.jyu.fi writes:
>On Tue, 30 Jun 1998, Roger Dingledine wrote:
>
>> My original thought there was to put a set of questions at the top that
>> would tell us which category this user was in. I guess I was hoping we'd
>> have enough from each category that we would in effect be having many
>> different surveys, with the same questions. (This means it's extremely
>> important to come up with questions that work for each 'type' of user.)
>
>It might be possible to write the beginning of the survey so that it saves
>different types of users of reading some big parts of survey, which is
>totally out point (for example too technical) to them? Or at least fix
>him/herself to a 'type' of user. So that we can interpret, which issues
>she/he negleted for which reason.

If we break the survey up into two parts, then we get several advantages.
The first section consists of a set of perhaps 10 choices of "user
profile", and the person picks one of them. This means that we can ask
many questions at once, so that part will go faster. Also, that means that
we won't be asking those questions on the survey itself, so we'll have
more space on the survey for the later questions (it'll seem like it has
fewer questions).

This seemed like a neat idea at first, but I'm having some problems with
it. I can't come up with many real examples ("windows user and professional
programmer"), and even then it doesn't have all the user profile questions
we were planning to ask. I don't see any good way to do it without asking
all the questions; from those we can run the answers through a little
script that outputs "yup, he's a windows user and a professional
programmer, give him the winuserprogrammer survey."

This isn't necessarily an argument against a two-phase survey, as long as
we make the first part short (perhaps a page and easy to fill out) so they
don't actually notice they're filling stuff out until they get to the next
part. The argument against a two-phase survey would be that they don't get
to go back and change user profile information after they've finished the
survey (or they don't get to take the parts in reverse order, etc).

>I agree that it is also 
>very important to find the right kind of questions to make different
>'type' of users... But how to do it? I was thinking that in the beginning
>we perhaps need to ask some quite big number of people to describe
>us typologies of users. Or can we invent them ourselves? This leads me to
>the following:
>
>What are the important issues to make differences
>of different type of users? 

This is a really tough question. You're asking two things here:

a) are these issues going to have an impact on our statistics?
b) so what? will we be able to draw any conclusions if they do or don't?

I think for the most part, question 'a' is one that the survey itself
is meant to answer. I have no clue what the answers to these questions are.
But I'd love to know. And I think once we have some responses, we'll get a
pretty good idea of how relevant each question is.

We can also use these questions as a guide for designing the rest of the
questions on the survey, and as a sanity check to figure out if the
questions we're asking are actually useful to us. These are excellent
questions to keep in mind.

>* Does it make an important difference if people use a computer at home or
>at work?  

It matters insofar as we need to figure out which user population we're
targetting, and we need to figure out what they require in their operating
environment.

>* Does it make an important difference, if people maintain/install the
>computer and the OS themselves or not?

Yes, because if most people we're targetting don't maintain/install the OS
themselves, then we should quit trying to make it so they can do it. (I
suspect the answer is that most people do it themselves. But I don't think
anybody really knows for sure around here.)

>* What about making difference between people, who make/don't make 
>decisions of purchasing the OS and/or the computer?
>
>* What about the differences of purchasing power (richness)?
>
>* Differences between the tasks the computer is used?
>
>* Differences between computers available for the installation?
> 
>I mean that is important to invent good/practical dimensions to work with.
>It probably *helps* nothing if we find out that there is difference
>between sexes how much each gender does programming for their OSs? Does it
>mean that we had to make the differences bigger or smaller?  



>Besides asking what is important what is not, we might need to ask as well
>what is possible and what is not? I think that there might be a quite big
>number of would-be-users, who have limitations of machinery for example,
>which puts limitations on the reality of an OS regardless how important
>some functionalities are? 

This is a good point. Shit. I was hoping to keep the list one-dimensional. :)
(That is, only asking one question about each issue, and it's the same
question all the way down.)

My argument against the "is it possible?" question is that end-users don't
necessarily have a good idea of what is possible and what isn't possible on
a given set of hardware. "I can't run a real multi-tasking operating system
on a 386" probably sounds like a good rule of thumb to a Windows user these
days, but it sounds like bunk to an experienced Linux user.

Is this a good enough rebuttal, or should I start trying to figure out a
good way to ask multiple questions of each issue?

>The general aim is to make a better, cheaper and freeer OS for anyone?
>That probably means very different things to different people: someone
>wants an OS, which he/she can her/himself develop, someone an OS, which
>doesn't need any fixing, just using?  
>
>The mythical 'end-user' means a person, who just wants a free mac? When it
>doesn't work he goes to a workshop to ask professionals to put it in order
>like an 'enduser' of a car, which stopped? Is the most important
>difference between the looked-for SEUL-user and an existing linux-user
>that the latter 'develops' the OS while the former doesn't? Computers are
>a means to the former and an end for the latter? 

Exactly. Check out the "Defining the End-User" section of our manifesto,
http://www.seul.org/what/manifesto.html to read a different wording of what
you just said. :)

>Actually I think that the
>meaning of the SEUL project is to find a way how an *enduser* can be a
>developer without writing programs him/herself but by telling
>programmers, which kind of programs are needed. Even that kind, which
>don't exist yet. We need a development process from down to up (from users
>point up to a programmer) contrary to the normal process from the
>programmer to the user.  
>
>I think that the big general problem of the technology of our age
>is that its maintenance is too far or out of the reach of the user. When
>machines don't work, the normal (end)user is helpless. Which type of on
>the reach control is needed? I don't think that a typical SEUL-user needed
>a possibility to write him/herself corrections to the scripts of the OS,
>but... what? A reach to a programmer? Or only an OS, which almost never
>crashes?   

It is the case that Linux has the unique opportunity of having a lot of
really responsive developers. It's also the case that these developers
are volunteers, so they only develop what they want to develop. 

The issue of enabling the user to contact the programmers in a useful
way is absolutely huge. The Linux User Support Team (at
http://www.ch4549.org/lust/) are starting to take a look at this;
they're going at it from a practical view rather than a theoretical
view. This is probably a different thread, though, and I'm going to
leave it alone for a couple weeks. :)

Actually, does this imply there should be another question on the
survey -- how important is it for you to be able to contact the
developers and tell them what you want, and how important is a
quick response on that?
 
>> It's certainly an interesting idea to just pick one of those types of
>> users, and ignore the rest. My concerns are:
>> * Which one(s) do we pick, and which do we ignore?
>> * Do we intend to cover the other ones later? How many chances do we
>>   get for sending out surveys, before people quit answering them?
>
>I don't think we could pick only some types of users. We just need to
>reach new kind of users besides already existing type of linux users. 
>It might help as well to find out the type of existing linux user and
>compare that to 'the normal computer user'? That survey could be made by
>using existing linux-discussion groups of the net?
>
>Might be the most important task in the beginning to get people to start
>answering... Or maybe even better asking questions, if this or that is
>possible or existing. I think that the changes to send the surveys are the
>more limited the less people have themselves possibilities to
>change/develop the survey itself. 

The problem with changing the survey after we've already given it to a lot
of people is that we can't easily combine (statistically) responses we got
from the first version with responses we get from the new version. My
instinct says this is a very important issue, but I can't really back that
up. Do other people have thoughts on this one?

>> My ideal case would be a survey that covers everybody and gives us a
>> lot of information about each type. But maybe we won't be able to get
>> that, it's true.
>
>> Ultimately, I'd like to get a quantitative survey. If that means getting
>> a qualitative survey first so we know what we should be asking, that works
>> too. I hope we can skip that step, though.
>
>We might be able to skip that step by doing this what we are doing right
>now. But do we have enough different types of people on this list now? We
>maybe needed some professional end-user instructors to know what are the
>general problems and expectations of people starting using computers? SEUL
>doesn't need to be just a combination of best propertions of all OSs but
>it could be based on that principle what people wanted to do with a
>computer? 

Ok. I've been doing a bit of advertising lately; I'll try to look
specifically for a couple people experienced with teaching Windows to
(lots of) end-user type people. Hopefully that will simplify things
a bit. But the short answer is: this survey is a combination of the best
properties of all OSs, but a distribution oriented towards an end-user
won't be. A distribution will take the answers from the survey and
figure out which parts are most important, and focus on those.

>Anotherkind of issue I have in my mind is: is it better to make a
>hypotethical SEUL-distribution ready, small, simple, true and
>nevercrashing or like a draft of not-ready-yet plans how to fulfill all
>the hopes of computing in the future (like linux-distributions tend to
>be)?

The former, clearly. :)

>> Can you expand on this further? What sort of 'method' did you have in mind?
>
>We might just try to ask all the people who answered the survey to add
>their own comments and questions they wanted to be asked in this survey.  
>It might be a start if all of us on this list just tried to find, lets say
>ten people each to check and aswer the survey in the beginning. Just try
>to think about all really different type of people we know using
>computers.

Ok. I'm going to come up with a plan for this, and I'm going to make it a
separate mail so people will actually read it. (Congratulations if you've
made it this far in this mail. :)

>Would it be possible to send somehow a free existing linux-distribution to
>everyone who completed the task... to send more programs to everyone who
>sent her/his opinion on the distribution he got... 

Some sort of incentive to take the survey is a good idea. I shall give
some more thought to it. I don't think we want to do anything that costs
money, because I don't think anybody wants to spend the money.

One thing I was planning was to give them the statistics of the survey
so far, so they know how things are going and can see the types of responses
other people are giving. But then, I'd want to be able to provide those
statistics even if you don't fill out the survey. I guess we'll be relying
on people filling it out from the goodness of their hearts. Which will skew
our survey results even farther towards people who already care about linux.

>Still I have in my mind an idea that somehow we needed to get people to
>try themselves the OS as well besides telling us how they wanted it to
>be...

Well, the survey isn't linux-centric. I agree that people ought to try using
a computer before filling out the survey. But I think most of them will have
done that. (On the other hand, if they've only used one operating system
(no matter what operating system that is), they won't recognize how useful
some of the issues we ask about really are. I don't know how to get around
that, though.)
 
>We might also seriously think about the geographical areas of the survey. 
>Besides America and Europe, Asia is a really would-be-important area.
>Especially India and China. In India we don't have even much
>language-problems, the problem is the availability of connections... But I
>have a dream: we could perhaps make the two most populous countries of the
>world, China and India, lands of linux. They are free and open for it.   

--Roger