Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit b25c402

Browse files
gvanrossumgvanrossum
committed
Initial revision
1 parent e944da8 commit b25c402

2 files changed

Lines changed: 368 additions & 0 deletions

File tree

‎Demo/scripts/newslist.doc‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
NEWSLIST
2+
========
3+
A program to assist HTTP browsing of newsgroups
4+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5+
6+
WWW browsers such as NCSA Mosaic allow the user to read newsgroup
7+
articles by specifying the group name in a URL eg 'news:comp.answers'.
8+
9+
To browse through many groups, though, (and there are several thousand
10+
of them) you really need a page or pages containing links to all the
11+
groups. There are some good ones out there, for example,
12+
13+
http://info.cern.ch/hypertext/DataSources/News/Groups/Overview.html
14+
15+
is the standard one at CERN, but it only shows the groups available there,
16+
which may be rather different from those available on your machine.
17+
18+
Newslist is a program which creates a hierarchy of pages for you based
19+
on the groups available from YOUR server. It is written in python - a
20+
splendid interpreted object-oriented language which I suggest you get
21+
right now from the directory /pub/python at ftp.cwi.nl, if you haven't
22+
already got it.
23+
24+
You should be able to see some sample output by looking at:
25+
http://pelican.cl.cam.ac.uk/newspage/root.html
26+
27+
Descriptions of newsgroups can be added from a file with one group
28+
per line. eg:
29+
30+
alt.foo Articles about foo
31+
comp.bar Programming in 'bar' and related languages
32+
33+
A suitable list detailing most groups can be found at ftp.uu.net in
34+
/uunet-info/newsgroups.gz.
35+
36+
Make sure you read the information at the beginning of the program source and
37+
configure the variables before running.
38+
39+
In addition to python, you need:
40+
41+
An NNTP-based news feed.
42+
A directory in which to put the pages.
43+
44+
The programming is not very beautiful, but it works! It comes with no
45+
warranty, express or implied, but with the hope that some others may
46+
find it useful.
47+
48+
Comments, improvements & suggestions welcomed.
49+
Quentin Stafford-Fraser
50+
51+
----------------------------------------------------------------------
52+
Quentin Stafford-Fraser
53+
http://pelican.cl.cam.ac.uk/people/qs101/me.html
54+
55+
Cambridge University Computer Lab Rank Xerox Cambridge EuroPARC
56+
[email protected] [email protected]
57+
Tel: +44 223 334411 Tel: +44 223 341521
58+
Fax: +44 223 334679 Fax: +44 223 341510
59+
----------------------------------------------------------------------

‎Demo/scripts/newslist.py‎

Lines changed: 309 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,309 @@
1+
#! /usr/local/bin/python
2+
#######################################################################
3+
# Newslist $Revision$
4+
#
5+
# Syntax:
6+
# newslist [ -a ]
7+
#
8+
# This is a program to create a directory full of HTML pages
9+
# which between them contain links to all the newsgroups available
10+
# on your server.
11+
#
12+
# The -a option causes a complete list of all groups to be read from
13+
# the server rather than just the ones which have appeared since last
14+
# execution. This recreates the local list from scratch. Use this on
15+
# the first invocation of the program.
16+
#
17+
# This assumes an NNTP news feed.
18+
#
19+
# Feel free to copy, distribute and modify this code for
20+
# non-commercial use. If you make any useful modifications, let me
21+
# know!
22+
#
23+
# (c) Quentin Stafford-Fraser 1994
24+
# [email protected] [email protected]
25+
# #
26+
#######################################################################
27+
import sys,nntplib, string, marshal, time, os, posix, string
28+
29+
#######################################################################
30+
# Check these variables before running! #
31+
32+
# Top directory.
33+
# Filenames which don't start with / are taken as being relative to this.
34+
##topdir='/anfs/qsbigdisc/web/html/newspage'
35+
topdir = '/hosts/buizerd/ufs/www/cwi/cwionly/newstree'
36+
37+
# The name of your NNTP host
38+
# eg.
39+
# newshost = 'nntp-serv.cam.ac.uk'
40+
# or use following to get the name from the NNTPSERVER environment
41+
# variable:
42+
##newshost = posix.environ['NNTPSERVER']
43+
newshost = 'charon.cwi.nl'
44+
45+
# The filename for a local cache of the newsgroup list
46+
treefile = 'grouptree'
47+
48+
# The filename for descriptions of newsgroups
49+
# I found a suitable one at ftp.uu.net in /uunet-info/newgroups.gz
50+
# You can set this to '' if you don't wish to use one.
51+
##descfile = 'newsgroups'
52+
descfile = '/usr/lib/news/newsgroups'
53+
54+
# The directory in which HTML pages should be created
55+
# eg.
56+
# pagedir = '/usr/local/lib/html/newspage'
57+
# pagedir = 'pages'
58+
pagedir = topdir
59+
60+
# The html prefix which will refer to this directory
61+
# eg.
62+
# httppref = '/newspage/',
63+
# or leave blank for relative links
64+
# between pages. (Recommended)
65+
httppref = ''
66+
67+
# The name of the 'root' news page in this directory.
68+
# A .html suffix will be added.
69+
##rootpage = 'root'
70+
rootpage = 'index'
71+
72+
# Set skipempty to 0 if you wish to see links to empty groups as well.
73+
# Only affects the -a option.
74+
##skipempty = 1
75+
skipempty = 0
76+
77+
# ---------------------------------------------------------------------
78+
# Less important personal preferences:
79+
80+
# Sublistsize controls the maximum number of items the will appear as
81+
# an indented sub-list before the whole thing is moved onto a different
82+
# page. The smaller this is, the more pages you will have, but the
83+
# shorter each will be.
84+
sublistsize = 4
85+
86+
# That should be all. #
87+
#######################################################################
88+
89+
from nntplib import NNTP
90+
from stat import *
91+
92+
rcsrev = '$Revision$'[11:15]
93+
desc = {}
94+
95+
# Make (possibly) relative filenames into absolute ones
96+
treefile = os.path.join(topdir,treefile)
97+
descfile = os.path.join(topdir,descfile)
98+
page = os.path.join(topdir,pagedir)
99+
100+
# First the bits for creating trees ---------------------------
101+
102+
# Addtotree creates/augments a tree from a list of group names
103+
def addtotree(tree, groups):
104+
print 'Updating tree...'
105+
for i in groups:
106+
parts = string.splitfields(i,'.')
107+
makeleaf(tree, parts)
108+
109+
# Makeleaf makes a leaf and the branch leading to it if necessary
110+
def makeleaf(tree,path):
111+
j = path[0]
112+
l = len(path)
113+
114+
if not tree.has_key(j):
115+
tree[j] = {}
116+
if l == 1:
117+
tree[j]['.'] = '.'
118+
if l > 1:
119+
makeleaf(tree[j],path[1:])
120+
121+
# Then the bits for outputting trees as pages ----------------
122+
123+
# Createpage creates an HTML file named <root>.html containing links
124+
# to those groups beginning with <root>.
125+
126+
def createpage(root, tree, p):
127+
filename = os.path.join(pagedir,root+'.html')
128+
if root == rootpage:
129+
detail = ''
130+
else:
131+
detail = ' under ' + root
132+
f = open(filename,'w')
133+
# f.write('Content-Type: text/html\n')
134+
f.write('<TITLE>Newsgroups available' + detail + '</TITLE>\n')
135+
f.write('<H1>Newsgroups available' + detail +'</H1>\n')
136+
f.write('<A HREF="'+httppref+rootpage+'.html">Back to top level</A><P>\n')
137+
printtree(f,tree,0,p)
138+
f.write('<I>This page automatically created by \'newslist\' v. '+rcsrev+'.')
139+
f.write(time.ctime(time.time()) + '</I><P>')
140+
f.close()
141+
142+
# Printtree prints the groups as a bulleted list. Groups with
143+
# more than <sublistsize> subgroups will be put on a separate page.
144+
# Other sets of subgroups are just indented.
145+
146+
def printtree(f, tree, indent, p):
147+
global desc
148+
l = len(tree)
149+
150+
if l > sublistsize and indent>0:
151+
# Create a new page and a link to it
152+
f.write('<LI><B><A HREF="'+httppref+p[1:]+'.html">')
153+
f.write(p[1:]+'.* ...')
154+
f.write('</A></B>\n')
155+
createpage(p[1:], tree, p)
156+
return
157+
158+
kl = tree.keys()
159+
160+
if l > 1:
161+
kl.sort()
162+
if indent > 0:
163+
# Create a sub-list
164+
f.write('<LI><B>'+p[1:]+'</B>\n<UL>')
165+
else:
166+
# Create a main list
167+
f.write('<UL>')
168+
indent = indent + 1
169+
170+
for i in kl:
171+
if i == '.':
172+
# Output a newsgroup
173+
f.write('<LI><A HREF="news:' + p[1:] + '">'+ p[1:] + '</A> ')
174+
if desc.has_key(p[1:]):
175+
f.write(' <I>'+desc[p[1:]]+'</I>\n')
176+
else:
177+
f.write('\n')
178+
else:
179+
# Output a hierarchy
180+
printtree(f,tree[i], indent, p+'.'+i)
181+
182+
if l > 1:
183+
f.write('\n</UL>')
184+
185+
# Reading descriptions file ---------------------------------------
186+
187+
# This returns an array mapping group name to its description
188+
189+
def readdesc():
190+
global desc
191+
192+
desc = {}
193+
194+
if descfile == '':
195+
return
196+
197+
try:
198+
d = open(descfile, 'r')
199+
print 'Reading descriptions...'
200+
except (IOError):
201+
print 'Failed to open description file ' + descfile
202+
return
203+
l = d.readline()
204+
while l != '':
205+
bits = string.split(l)
206+
try:
207+
grp = bits[0]
208+
dsc = string.join(bits[1:])
209+
if len(dsc)>1:
210+
desc[grp] = dsc
211+
except (IndexError):
212+
pass
213+
l = d.readline()
214+
215+
# Now the main program --------------------------------------------
216+
217+
def main():
218+
global desc
219+
220+
connected = 0
221+
tree={}
222+
223+
# Check that the output directory exists
224+
if not os.path.isdir(pagedir):
225+
print 'Directory '+pagedir+' does not exist.'
226+
print 'Shall I create it for you? (y/n)'
227+
if sys.stdin.readline()[0] == 'y':
228+
try:
229+
os.mkdir(pagedir,0777)
230+
except:
231+
print 'Sorry - failed!'
232+
sys.exit(1)
233+
else:
234+
print 'OK. Exiting.'
235+
sys.exit(1)
236+
237+
try:
238+
print 'Connecting to '+newshost+'...'
239+
if sys.version[0] == '0':
240+
s = NNTP.init(newshost)
241+
else:
242+
s = NNTP(newshost)
243+
connected = 1
244+
except (nntplib.error_temp, nntplib.error_perm), x:
245+
print 'Error connecting to host:'
246+
print x
247+
print 'I\'ll try to use just the local list.'
248+
249+
# If -a is specified, read the full list of groups from server
250+
if connected and len(sys.argv) > 1 and sys.argv[1] == '-a':
251+
print 'Getting list of all groups...'
252+
treedate='010101'
253+
info = s.list()[1]
254+
groups = []
255+
print 'Processing...'
256+
if skipempty:
257+
print '\nIgnoring following empty groups:'
258+
for i in info:
259+
if skipempty and string.atoi(i[1]) < string.atoi(i[2]):
260+
print i[0]+' ',
261+
else:
262+
groups.append(i[0])
263+
print '\n(End of empty groups)'
264+
265+
# Otherwise just read groups created since local file last modified.
266+
else:
267+
print 'Reading current local group list...'
268+
try:
269+
treetime = time.localtime(os.stat(treefile)[ST_MTIME])
270+
except:
271+
print '\n*** Failed to open local group cache '+treefile
272+
print 'If this is the first time you have run newslist, then'
273+
print 'use the -a option to create it.'
274+
sys.exit(1)
275+
treedate = '%02d%02d%02d' % (treetime[0] % 100 ,treetime[1], treetime[2])
276+
try:
277+
dump = open(treefile,'r')
278+
tree = marshal.load(dump)
279+
dump.close()
280+
except (IOError):
281+
print 'Cannot open local group list ' + treefile
282+
283+
if connected:
284+
print 'Getting list of new groups since start of '+treedate+'...',
285+
groups = s.newgroups(treedate,'000001')[1]
286+
print 'got '+`len(groups)`+'.'
287+
288+
if connected:
289+
addtotree(tree, groups)
290+
try:
291+
dump = open(treefile,'w')
292+
groups = marshal.dump(tree,dump)
293+
dump.close()
294+
print 'Saved list to '+treefile+'\n'
295+
except:
296+
print 'Sorry - failed to write to local group cache '+treefile
297+
print 'Does it (or its directory) have the correct permissions?'
298+
sys.exit(1)
299+
300+
readdesc()
301+
print 'Creating pages...'
302+
createpage(rootpage, tree, '')
303+
print 'Done'
304+
305+
306+
main()
307+
308+
# That's all folks
309+
######################################################################

0 commit comments

Comments
 (0)