Re: Arm Release 1.4.0

[Damian Johnson <atagar1@xxxxxxxxx>]

> I mainly see the problem when scrolling through the connection results:

Ahhh, gotcha. The problem here isn't really the connection resolution
then, but rather the sucky panel implementation. The old code had a
habit of doing lengthy operations in the draw loop rather than using
cached results, so most likely the high cpu usage is from sort
operations being triggered each time it scrolls. Making the
sortConnections function (line 835) a no-op (just return right away)
might address this.

That said, here's a couple of observations about the arm cpu field:
- Scrolling through the connections is the worst, but the newer panels
exhibit cpu spikes too. On my system scrolling the log goes to 7% and
scrolling the config listing goes to around 35%. I'll look into this
some, but I'm suspecting that it may be unavoidable since moving the
cursor requires a panel refresh so scrolling a dozen times a second is
always gonna require a dozen refreshes. The only thing to rate limit
scrolling is the redraw, so holding down the scroll key in essence
creates an open loop where arm is scrolling and redrawing as fast as
it can (maybe I should try to coalesce those events...).

In essence this means that the performance improvements that I've been
making in those panels makes it so users can scroll faster, but
doesn't really improve the cpu usage when users hold the arrow keys
down. Not quite what I'd intended. :P

- I'm using the os.times() function to get the arm cpu usage which is
*supposed* to include child processes. However, it doesn't seem to be
including the ps and netstat lookups since disabling them or making
them occur more frequently doesn't have an effect on this value. Pity,
that was the main point in including this stat...

Per chance do you have any ideas for how to include the system calls
in this figure?

> I get the impression that the back-off mechanism doesn't apply in
> that situation?

Backoff only applies if the connection results are taking too long.
The netstat runtime was 0.0587 seconds so backoff was probably being
triggered a little bit, but wouldn't help with the cpu usage being
wasted on the sorting.

> Another problem I noticed that may or may not be connection related
> is that arm occasionally stops working and neither updates the screen
> nor reacts to keyboard input.

Uggg, that's annoying. The low cpu usage means that it's probably not
falling into any sort of busy wait situation. How frequently does it
occur? Does it always happen when on a particular part of the
interface? Does this arise when interacting with arm, or does it occur
on its own (ie, not in response to user input)? A repro case would be
very helpful...

For debugging you can run arm at its debug runlevel (arm -e 1) with
the "features.logFile" set to collect arm's events and see if it's
continuing to do anything while it seems frozen. Also, to double check
that ssh isn't the issue you could run arm in a detached screen
session, then reattach with another ssh connection when it seems
frozen to see if they get the same results.

> Arm Test Options: ...

Great! Did you run the test multiple times? It might have been biased
due to cached results (favoring netstat since it had been running in
arm). However, besides lsof outperforming sockstat that was more or
less what I'd expected. Thanks!

I should have the testing utility fetch twice to account for this...

> If the output of the resolver utility is (partly) localized
> the grep for "ESTABLISHED" no longer matches.

Ack! Would you suggest modifying LANG, letting it fall back to another
resolver, or something else? The resolver fallback in this case isn't
ideal since (a) netstat tends to outperform the others and (b) since
the netstat command is available but failing it'll make three attempts
before going to the fallback (so fifteen seconds before the
resolutions appear).

If you have an idea for how to modify the LANG env variable safely
(ie, without unintended consequences on other platforms nor leaving
the user's terminal in a bad state, even when arm crashes) then I'd be
happy to apply a patch. I don't trust that I have a deep enough
understanding of this to do the above properly. :(

That said, if this is a Linux-only issue then it might be moot if the
connection resolution via /proc contents works.

> It works, thanks.

Hazaa!

> Hmm, I thought the confusing controller case should be handled
> with the ipAddressIsPrivate() patch

Oops. You're right, that shouldn't be happening. That was probably due
to something I did while messing with the patch.

Out of curiosity, what is going to be involved with the bsd port? When
arm makes a new release what is the process for getting the port
upgraded? Emailing you? Filing a ticket?

Also, if this is going to require a permanent tarball link (like
Gentoo) then please let me know and I'll add it to the site.

Cheers! -Damian