LIST NOTICE: JavaScript Mayhem (Technical Morass; Long)

[Richard Bullington <rbulling@obscure.org>]

-----------Short Version-----------
The Obscure Organization's 28.8kbps link to the 'net is saturated, due to
the JavaScript mailing list, and I need help from a few JavaScript list
members to fix this. Donating some spare machine cycles and bandwidth for
mail delivery can save the list. 

Until I can resolve the bandwidth problems, I am moderating the list and
halting discussion until we find a solution. Sorry, but I can't get my
for-pay work done with the line this slammed.

----------Long Version--------------

Hello. Sorry to interrupt the party, but...

My name is Richard Bullington. I run The Obscure Organization, a small 
non-profit community networking initiative. Among other things, we host
the JavaScript list.

Unfortunately for Obscure, the list has been growing and growing with no 
end in sight. The main server runs Linux 1.2.13 on an ISA-bus 486DX-33 with 
32MB RAM, and 1.6 GB of SCSI disk space. 

I have had dozens of sendmail processes chewing up my memory and 
swap space, pushing 30MB of active memory into swap. 

Obscure's connection to the Internet runs over a 28.8kbps modem. Look at
http://www.obscure.org/statistics/traffic/ for a graph of the network
traffic. Note that the midpoint is 28.8kbps, and it had been bursting well
above that for several hour periods. The statistics have been broken since
I upgraded to Slackware 3.0. Using statnet, I have manually measured my
PPP connection running at about 4KBps (32Kbps), bursting often to 8.75KBps
(70Kbps!!!). V.42bis compression really does work :)

I am starting to hit a wall, where the PPP link is bursting above 
28.8Kbps for a good portion of the day. Yesterday, Obscure sent out over 
30,000 mail messages, and queued another 10,000:

 Syslog   Input:  (total)     Output Statistics:
File Date  Msgs Kbytes AvgSz    Sent AvgDelay Dferd Que'd Other Local   SMTP
   Mar 21   765   1730  2316   36019 05:08:11   137  9873   138  1118  33916
   Mar 20   908   1509  1702   21031 02:44:42    88   727    66  1042  19212

(This is from syslog-stat.pl, which I have modified heavily. If there is 
interest, I will create a web page that reveals all the tools I used to 
diagnose this mail nightmare...)

The delays are getting larger and larger. March 21st represents about 
62MB of outgoing mail. This means that the JavaScript list has a 
sustained transfer rate of about 5Kbps. This would be nothing over ISDN, 
or a T1 line, but it digs deep into a 28.8Kbps connection.

There are several ways out of this mess:
 
 * Obscure can get a faster connection (I need $350 per month more to 
   jump to full-time 128K ISDN. I don't expect anyone on this list to 
   contribute enough to make a difference here).
 * The JavaScript list can move elsewhere or die completely.
 * JavaScript can be moderated by someone with enough time to do it right,
and, the one that I'd like to try,
 * The generous people on the JavaScript list can help me out by 
   providing mail exploders for the various top-level domains and regions.

To provide a mail exploder, all you need to do is have a relatively fast
and highly reliable machine (i.e. it is always running) with Sendmail V8
that can take some additional mail delivery load. A machine that already
acts as a mail hub for a medium sized organization is ideal. The machine
should have fast (128K+) connectivity to the 'Net and good links to
neighboring countries where appropriate (see domain breakdown at the end
of this message). 

If you would like to volunteer for this, please let me know. You don't 
need to do a thing to your systems, just give me the address of the 
machine running sendmail, what domain or domains you want to act as a 
hub for, and a brief description of the mail hub machine.  You probably 
won't even notice the load. 

This rule, when landed in Ruleset 0 of the sendmail configuration, will 
route mail for a particular top-level domain to a host running sendmail 
for final delivery:  (Thanks to Jeff Uphoff and the linux-security list)

R$*<@$+.domain.>$*      $#esmtp $@router.host $:$1<@$2.domain.>$3

So, for all mail coming from Obscure to .com adddresses, the rule is:

R$*<@$+.com.>$*      $#esmtp $@some.generous.soul.com $:$1<@$2.com.>$3

The vast majority (>90%) of the mail traffic from Obscure is for the
JavaScript list. As far as I know, this solution will direct all mail
traffic from Obscure to the mail gateways for each domain. 

Here is how many people are on the lists: 

    560    javascript
    688    javascript-digest
   1248    total

I wrote a small perl program that counts the top-level domains of
the email addresses. I have used the results to produce the following tables:

Here is a breakdown of top level domains by subscriber. I figure that we 
need an exploders for at least 75% of these for the list to survive 
unmoderated.

Top Level Domains       Aggregate Number of Subscribers
com                     506
net                     198
edu                     146
it de fr dk be at ch su 97
us ca org mil gov mx    64
jp kr hk sg tw my th    50
uk gb ie es pl          40
se no fi                40
au nz                   24

TOTAL                  1248 

This breakdown covers about 95% of all the addressees. Here is the full 
breakdown by top level domain:

Top level domains:

506	com
198	net
146	edu
31	it
25	uk
24	de
24	ca
19	org
19	jp
19	au
17	se
15	no
15	nl
13	kr
13	fr
12	br
11	us
11	be
8	mil
8	gov
8	fi
8	es
8	dk
6	sg
6	at
5	nz
5	mx
5	hk
5	ch
4	su
4	pt
4	my
4	ie
3	za
3	tw
3	th
3	pl
3	hu
3	cl
2	tt
2	tr
2	pe
2	il
2	gr
2	cz
2	ar
1	ve
1	lu
1	kw
1	in
1	id
1	gb
1	ee
1	d
1	ae


Phew! If you made it all the way here, you have a lot of endurance. Sorry 
for the long post, but I have been very frustrated, not being able to 
effectively telecommute and get my real job done because of this.

Richard Bullington  <rbulling@obscure.org> http://www.obscure.org/
Keeper of The Obscure Organization, wearing his JavaScript admin hat.