Re: gEDA-user: Collaborative Development of Boards

[gedau@xxxxxxxxxxxxx]

Hello Markus,

On Sun, Jan 16, 2011 at 10:11:31PM +0100, Markus Hitter wrote:
> Hi all,
>
> having run an open source electronics project with the intention of  
> collaborative development for a few months now, the experience is less 
> than inspiring. So I'm looking for opinions on how to do better.
>
> http://reprap.org/wiki/Generation_7_Electronics
> http://github.com/Traumflug/Generation_7_Electronics
>
> Looking at what happened in these few months, I think the essence can be 
> described in 3 topics:

Mech and programming bakground, I have the same, but I don't agree these
things would work any better in programming. As others, I also think 
these problems are basically the same for free and for non-free jobs, as 
others on the list already expressed. Team work is hard to do right, and 
this is largely independent of the field you try to do it in 
engineering.

>
> 1. Files aren't mergeable.
>
> While PCB isn't that bad at keeping changes to the saved file small,  
> there's always at least the also stored file creation date letting  
> merges fail. One can store a design's files in a Git repository, of  
> course, but always only in a linear fashion. As soon as one collaborator 
> works on something, all others have to wait until he's done. The 
> versioning system would actually need a locking mechanism, like good ol' 
> CVS had.

Yes, I agree on this one. I think storing such modification date is just 
the wrong thing to do. Since I started to use PCB, I always stored 
everything in svn and I often had minor problems with the date lines. 
Subversion, and any other decent version control system offers means to 
generate last mod date, author info, etc by the VCS in a way this 
meta-info will look like being part of the file content to the end user 
while it won't interfere with versioning, especially won't cause 
diffs.

I suggest we make this part of the file format optional, default turned 
off. (Feature request #1)

The other problem is diff. Again, I don't think there is anessential 
difference between team work and VCS in software development and in 
using the same methods/tools on other fields; at least we have the same 
methods, but some tools are partially missing. How does it work with 
software?

A. the source code is an ordered list of objects (not in the OOP sense). 
If you edit one object, that won't ever move other objects around by 
side effect. VCS systems I know depend on this feature. I think PCB 
already provides this feature, keeping order of objects, but in my 
daytime job I often meet non-programming tasks where the editor doesn't 
and if you do a minimal change, fix a typo in a text or anything alike, 
things get reordered and you always get a full-file-diff. So whatever 
tool you use, you must make sure it does minimal change needed in the 
source file.

B. diff. When you work on souce code, you make a diff to see what others 
changed, or what you are going to submit as your change. With 
programming, you have the same stages as with PCB. For programming:
 - real representation is when your program is running; this is the
   final form, as interpreted (not in the CS sense) by the machine, the 
   goal of your efforts
 - the source code is something abstract you don't see working while 
   editing; you look at your source code and interpret it, and imagine 
   how it would work, but sure it won't translate 1:1, you use your 
   brain a lot for that transformation while coding
 - diff is the second abstraction, a language describes changes between 
   2 such abstract source code. Experienced programmers have the skill 
   to read an intrepret the change, reconstruct the source code in head, 
   interpret those code and reconstruct the final behaviour. This 
   requires jumping 2 levels of sbastraction in one step, transarently.

Now for a PCB (assuming we are talking about the layout work only, as 
your mail suggested):
 - real representation is what the GUI shows, this is almost 1:1 what 
   you will get in form of copper and plastic. 
 - the source code is PCB file, this is what fed into the VCS system. 
   Same abstraction as with program source code vs. running the program, 
   and PCB users can even do the same trick interpreting the sources to 
   different degree. Some of the most hardcore ones are even do 
   modifications using text editor on regular basis, and it is often 
   faster than ivoking PCB for a click-around session
 - you have the same diff tool, and the language is the same, so you 
   jump the same 2 level of abstrsctions.

So what's the big deal the two processes still differ so much seamingly? 
The middle one. Non-programming EEs usually won't do the middle step, 
which means they won't be able to do the 2nd abstraction either. Without 
diff, your VCS reduces to a shared file system with backups. 

In reality you probably won't ever get all your contributors to learn the file 
format and track changes on that level, so you'll need something else. 
Probably a graphical diff, that loads both PCBs, shows the common parts 
in grey, mismatching parts in inormla color (feature request #2). Well, 
this is much harder than first seems - you need to handle things like 
the board was moved around, etc. Maybe some sort of adding UUID to 
objects and keeping the UUID on move, rotate, etc would help (feature 
request #3).

Btw, this is orthogonal not only for programming vs. PCB, but for paying 
vs. free work. I happen to work for a small company of a large company 
group, so I see all sort of different team works. If people are not 
motivated in doing real good team work, whatever you do, paying or hobby 
project, team work will just fail. 

> 2. Agreements on design decisions are impossible
>
> For example connectors: Some use connectors with 4 mm spacing, others  
> use 5.08 mm spacing, third people consider anything pluggable as stupid 
> and use nothing but screw connectors. A project leader - y'know, open 
> source tries to avoid such terms - can do a decision, of course, but 
> single people will plainly refuse to manufacture or use anything without 
> their favoured connector. Such design details are apparently hardcoded in 
> electronics' people's brains.
>
> As a result, lots of incompatible designs exist, and there's nothing  
> like a "preprocessor", which could switch between different details on 
> the fly. The same applies for FETs, diodes, jumpers, whatever.

This is also orthogonal to electronics vs. anything-else. There are 
several different ways doing it right. Let me list 4 I prefer to use:
- Let the team consist only those who really fully agree on goals, 
naturally. Not because you force them to, but because they think alike. 
This is the hardest, takes a lot of investment in selecting the members, 
and a lot of hard choices as in this schema the team needs to kick out 
anyone who starts to diverge in the goals. No leader needed.
- Strong central coordination. Explicitly named leader(s) needed. The 
team should sit together to make the decisions together, but at the end, 
on a tie, the leader decides, or if no tie, the leader makes the 
decisions documented and explicit. I can't stress enough how important 
it is to get these decisions and the whole schema explicit. Especially 
for people looking for joining the efforts. Btw, having a formal leader 
does _not_ mean the decisions were not made in a democtratic way or the 
leader decides alone. It just guarantees however the community made the 
decisions, all members must follow.
- Have a main direction of development maintained by one of the above 
methods, and let contributors maintain their own branches slightly 
differing.
- Split up the work to as many small parts as many contributors you 
have, (let them) define clear and explicit interfaces between those 
parts. This way everyone works only on his own part except for the 
interfaces, which is done in real team work.

My experience is that doing one of the first two is extremely efficient 
with a good team: if you have 4 members working at 100%, the result 
won't be 400% but usually much more. This effect can not be reproduced 
with the other two methods. The main difference is whether you work 
really together, on the same project (same questions, same design, 
same file) or just work side by side and then put the results together 
at the end. The latter one don't have performance boost, only sum of 
individual performances in my experience.

Another method very fashionable nowdays with distributed version 
control systems is to let everyone have his own branch to work with. 
This seemingly makes them work on the same project, same files, etc, but 
in my experience at the end you get the same problem not having the 
extra boost.

(Of course, the boost works only with a _good_ team even if you use the 
right methods)

I don't think any new feature in the software would really help any in 
these.

> 3. No focus on the problem to solve
>
> If you look at the recent commits of this project you'll see  
> enhancements always coming along with a plentitude of unrelated changes. 
> Yes, make these pads a bit bigger, but also move a track here and change 
> a text there. "looks better".
>
> IMHO, doing such random changes is good for nothing but asking for  
> trouble. Yet, none of the coworkers seems to see what I'm talking about. 
> They do "looks better" all the time, making reviews a lot harder, and 
> sometimes you even get regressions.

I partly agree. Such cleanup sessions are very useful once you have 
everything ready and working, as separate series of commits. Actually 
the huge software projects which tend to get unusable over the years 
usually do so exactly because lack of such cleanup releases. You may 
also have friends complaining about specific GUI browsers slowly eating 
up all ram if ran for weeks. Why? Because each new release brings n+1 
new features and random bugfixes, but there's never a version where the 
changelog says "Deleted 20 unused features. Patched 300 memory leaks. No 
new features added". If you simply refuse any cleanup for a PCB, if the 
desing process is an ongoing one for many years and you have many 
versions, you will probably have the same problems in smaller scale.

However, your point is totally valid on the other hand: these little 
fixes, getting the silk layer look nicer for example, does not make any 
sense if you will move half of the elements in the near future, for 
example because you don't even have a decision on the connectors yet.

Here I suggest what I already suggested in section 2, and what others 
suggested in their mail: document your decisions and plans, and make 
them explicit to everyone joining the project. Make a roadmap, put it in 
a file in the root directory of the project. Tell newcomers we are at 
point 4 on that roadmap, so we simply do not accept patches dealing with 
point 5, 6, 7 or 8. 

Btw, documenting all these oftem triggers critics, like "too much talk, 
no action" or the documentation "gets too verbose", often "uninteresting 
details [of the development process]". In my experience this is normal, 
and shouldn't make you doubt the usefulness of this kind of 
documentation. It's more about some people don't like to think far ahead 
or make longer term plans, while others do; in a free development the 
duty of setting up those files and agreements are of those of the second 
group, while those who would just jump on it, start developing and "we 
will see" will not spend much time on such administration. Don't get me 
wrong, both types are important for the project, you just need to 
arrange things everyone works on his type of tasks.

Regards,

Tibor 


_______________________________________________
geda-user mailing list
geda-user@xxxxxxxxxxxxxx
http://www.seul.org/cgi-bin/mailman/listinfo/geda-user