Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit b60f2d0

Browse files
eric-s-raymonderic-s-raymond
committed
Framework code for compilerlike scripts.
1 parent 81937a4 commit b60f2d0

2 files changed

Lines changed: 241 additions & 0 deletions

File tree

‎Doc/lib/libcompilerlike.tex‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
\section{\module{compilerlike} ---
2+
framework code for building compiler-like programs.}
3+
4+
\declaremodule{standard}{set}
5+
\modulesynopsis{Framework code for building compiler-like programs.}
6+
\moduleauthor{Eric S. Raymond}{[email protected]}
7+
\sectionauthor{Eric S. Raymond}{[email protected]}
8+
9+
There is a common `compiler-like' pattern in Unix scripts which is useful
10+
for translation utilities of all sorts. A program following this pattern
11+
behaves as a filter when no argument files are specified on the command
12+
line, but otherwise transforms each file individually into a corresponding
13+
output file.
14+
15+
The \function{filefilter}, \function{linefilter}, and
16+
\function{sponge} functions in this module provide a framework and
17+
glue code to make such programs easy to write. You supply a function
18+
to massage the file data; depending on which entry point you use, it
19+
can take input and output file pointers, or it can take a string
20+
consisting of the entire file's data and return a replacement, or it
21+
can take in succession strings consisting of each of the file's lines
22+
and return a translated line for each.
23+
24+
All three of these entry points take a name, an argument list of files,
25+
a data transformation function, and a name transformation function.
26+
They differ only in the arguments they pass to the transformation
27+
function when it is called.
28+
29+
The name argument is not used by the functions in this module, it is
30+
simply passed as the first argument to the transformation function.
31+
Typically it is a string that names the filter and is used in
32+
generating error messages, but it could be arbitrary data.
33+
34+
The second argument, is interpreted as a list of filenames. The files
35+
are transformed in left to right order in the list. A filename
36+
consisting of a dash is interpreted as a directive to read from
37+
standard input (this can be useful in pipelines).
38+
39+
The third argument is the data transformation function.
40+
Interpretation of this argument varies across the three
41+
entry points and is described below.
42+
43+
The fourth, optional argument is a name transformation function or
44+
name suffix string. If it is of string type, the shortest suffix of each
45+
filename beginning with the first character of the argument string
46+
is stripped off. If the first character of the argument does not
47+
occur in the filename, no suffix is removed. Then the name suffix
48+
argument is concatenated to the end of the stripped filename. (Thus,
49+
a name suffix argument of ".x" will cause the filenames foo.c and
50+
bar.d to be transformed to foo.x and bar.x respectively.)
51+
52+
If the fourth argument is specified and is a function, the name of the
53+
input file is passed to it and the return value of the function
54+
becomes the name of the output software. If this argument is not
55+
specified, the imnput file is replaced with the transformed version.
56+
57+
Replacement of each file is atomic and doesn't occur until the
58+
translation of that file has completed. Any tempfiles are removed
59+
automatically on any exception thrown by the translation function,
60+
and the exception is then passed upwards.
61+
62+
\begin{funcdesc}{filefilter}{name, arguments, trans_data\optional{,trans_file}}
63+
Filter using a function taking the name and two file-object
64+
arguments. The function is expected to read data from the input file
65+
object, transform it, and write the data to the output file object.
66+
When the function terminates, the translation is done. The return
67+
value of the transformation function is not used.
68+
\end{funcdesc}
69+
70+
\begin{funcdesc}{linefilter}{name,arguments,trans_data\optional{,trans_file}}
71+
Filter using a function taking the name and a string argument. The return
72+
value of the function should be a string. This function is applied to
73+
each line in the input file in turn; the return values become the
74+
lines of the transformed file.
75+
\end{funcdesc}
76+
77+
\begin{funcdesc}{sponge}{name, arguments, trans_data\optional{, trans_file}}
78+
Filter using a function taking the name and a string argument. The
79+
return value of the function should be a string. The function will be
80+
passed the entire contents of the input file as a string. The string
81+
return value of the function will become the entire contents of the
82+
transformed file.
83+
\end{funcdesc}
84+
85+
# End
86+
87+

‎Lib/compilerlike.py‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
"""
2+
compilerlike -- framework code for building compiler-like programs.
3+
4+
There is a common `compiler-like' pattern in Unix scripts which is useful
5+
for translation utilities of all sorts. A program following this pattern
6+
behaves as a filter when no argument files are specified on the command
7+
line, but otherwise transforms each file individually into a corresponding
8+
output file.
9+
10+
This module provides framework and glue code to make such programs easy
11+
to write. You supply a function to massage the file data; depending
12+
on which entry point you use, it can take input and output file pointers,
13+
or it can take a string consisting of the entire file's data and return
14+
a replacement, or it can take in succession strings consisting of each
15+
of the file's lines and return a translated line for each.
16+
17+
Argument files are transformed in left to right order in the argument list.
18+
A filename consisting of a dash is interpreted as a directive to read from
19+
standard input (this can be useful in pipelines).
20+
21+
Replacement of each file is atomic and doesn't occur until the
22+
translation of that file has completed. Any tempfiles are removed
23+
automatically on any exception thrown by the translation function,
24+
and the exception is then passed upwards.
25+
"""
26+
27+
# Requires Python 2.
28+
from __future__ import nested_scopes
29+
30+
import sys, os, filecmp, traceback
31+
32+
def filefilter(name, arguments, trans_data, trans_filename=None):
33+
"Filter stdin to stdout, or file arguments to renamed files."
34+
if not arguments:
35+
trans_data("stdin", sys.stdin, sys.stdout)
36+
else:
37+
for file in arguments:
38+
if file == '-': # - is conventional for stdin
39+
file = "stdin"
40+
infp = sys.stdin
41+
else:
42+
infp = open(file)
43+
tempfile = file + ".~%s-%d~" % (name, os.getpid())
44+
outfp = open(tempfile, "w")
45+
try:
46+
trans_data(file, infp, outfp)
47+
except:
48+
os.remove(tempfile)
49+
# Pass the exception upwards
50+
(exc_type, exc_value, exc_traceback) = sys.exc_info()
51+
raise exc_type, exc_value, exc_traceback
52+
if filecmp.cmp(file, tempfile):
53+
os.remove(tempfile)
54+
else:
55+
if not trans_filename:
56+
os.rename(tempfile, file)
57+
elif type(trans_filename) == type(""):
58+
i = file.rfind(trans_filename[0])
59+
if i > -1:
60+
file = file[:i]
61+
os.rename(tempfile, stem + trans_filename)
62+
else:
63+
os.rename(tempfile, trans_filename(file))
64+
65+
def line_by_line(name, infp, outfp, translate_line):
66+
"Hook to do line-by-line translation for filters."
67+
while 1:
68+
line = infp.readline()
69+
if line == "":
70+
break
71+
elif line: # None returns are skipped
72+
outfp.write(translate_line(name, line))
73+
74+
def linefilter(name, arguments, trans_data, trans_filename=None):
75+
"Filter framework for line-by-line transformation."
76+
return filefilter(name,
77+
arguments,
78+
lambda name, infp, outfp: line_by_line(name, infp, outfp, trans_data),
79+
trans_filename)
80+
81+
def sponge(name, arguments, trans_data, trans_filename=None):
82+
"Read input sources entire and transform them in memory."
83+
if not arguments:
84+
sys.stdout.write(trans_data(name, sys.stdin.read()))
85+
else:
86+
for file in arguments:
87+
infp = open(file)
88+
indoc = infp.read()
89+
infp.close()
90+
tempfile = file + ".~%s-%d~" % (name, os.getpid())
91+
try:
92+
outfp = open(tempfile, "w")
93+
except OSError:
94+
sys.stderr.write("%s: can't open tempfile" % name)
95+
return 1
96+
try:
97+
outdoc = trans_data(name, indoc)
98+
except:
99+
os.remove(tempfile)
100+
# Pass the exception upwards
101+
(exc_type, exc_value, exc_traceback) = sys.exc_info()
102+
raise exc_type, exc_value, exc_traceback
103+
if outdoc == indoc:
104+
os.remove(tempfile)
105+
else:
106+
outfp.write(outdoc)
107+
if not trans_filename:
108+
os.rename(tempfile, file)
109+
elif type(trans_filename) == type(""):
110+
i = file.rfind(trans_filename[0])
111+
if i > -1:
112+
file = file[:i]
113+
os.rename(tempfile, file + trans_filename)
114+
else:
115+
os.rename(tempfile, trans_filename(file))
116+
117+
if __name__ == '__main__':
118+
import getopt
119+
120+
def nametrans(name):
121+
return name + ".out"
122+
123+
def filefilter_test(name, infp, outfp):
124+
"Test hook for filefilter entry point -- put dashes before blank lines."
125+
while 1:
126+
line = infp.readline()
127+
if not line:
128+
break
129+
if line == "\n":
130+
outfp.write("------------------------------------------\n")
131+
outfp.write(line)
132+
133+
def linefilter_test(name, data):
134+
"Test hook for linefilter entry point -- wrap lines in brackets."
135+
return "<" + data[:-1] + ">\n"
136+
137+
def sponge_test(name, data):
138+
"Test hook for the sponge entry point -- reverse file lines."
139+
lines = data.split("\n")
140+
lines.reverse()
141+
return "\n".join(lines)
142+
143+
(options, arguments) = getopt.getopt(sys.argv[1:], "fls")
144+
for (switch, val) in options:
145+
if switch == '-f':
146+
filefilter("filefilter_test", arguments, filefilter_test,nametrans)
147+
elif switch == '-l':
148+
linefilter("linefilter_test", arguments, linefilter_test,nametrans)
149+
elif switch == '-s':
150+
sponge("sponge_test", arguments, sponge_test, ".foo")
151+
else:
152+
print "Unknown option."
153+
154+
# End

0 commit comments

Comments
 (0)