Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 62060f8

Browse files
vazirimvazirim
authored
gsm8k plan with few-shots (#870)
Signed-off-by: Mandana Vaziri <[email protected]>
1 parent 3910c3d commit 62060f8

File tree

2 files changed

+204
-0
lines changed

2 files changed

+204
-0
lines changed

‎examples/gsm8k/demos.yaml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
- |
2+
Problem:
3+
Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?
4+
5+
Plan:
6+
Figure out how many clips Natalia sold in May, by halving the number sold in April. Then add the number sold in April with the number sold in May.
7+
8+
- |
9+
Problem:
10+
Weng earns $12 an hour for babysitting. Yesterday, she just did 50 minutes of babysitting. How much did she earn?
11+
12+
Plan:
13+
First calculate how much Weng earns per minute. Then multiply this number by 50.
14+
15+
- |
16+
Problem:
17+
Betty is saving money for a new wallet which costs $100. Betty has only half of the money she needs. Her parents decided to give her $15 for that purpose, and her grandparents twice as much as her parents. How much more money does Betty need to buy the wallet?
18+
19+
Plan:
20+
First calculate how much money Betty already has, which is half of $100. Then calculate how much her grandparents gave her by multiplying how much her parents give her by 2. Calculate the difference between 100 and all the money she has and is given.
21+
22+
- |
23+
Problem:
24+
Julie is reading a 120-page book. Yesterday, she was able to read 12 pages and today, she read twice as many pages as yesterday. If she wants to read half of the remaining pages tomorrow, how many pages should she read?
25+
26+
Plan:
27+
First calculate how many pages Julie reads today by multiplying 12 by 2. Second, calculate the total read for yesterday and today. Third, calculate the number of remaining pages. Finally compute half the remaining pages.
28+
29+
- |
30+
Problem:
31+
James writes a 3-page letter to 2 different friends twice a week. How many pages does he write a year?
32+
33+
Plan:
34+
First calculate how many pages he writes to each friend every week. Second calculate the total number of pages he writes each week. Multiply that number by 52 to obtain how many pages he writes every month.
35+
36+
- |
37+
Problem:
38+
Mark has a garden with flowers. He Planted Plants of three different colors in it. Ten of them are yellow, and there are 80% more of those in purple. There are only 25% as many green flowers as there are yellow and purple flowers. How many flowers does Mark have in his garden?
39+
40+
Plan:
41+
First calculate how many more purple flowers there are. Second calculate how many purples flowers. Calculate the number of purple and yellow flowers together. Calculate the number of green flowers, knowing that it's 25% of purple and yellow flowers. Add the number of purple and yellow with the number of green flowers.
42+
43+
- |
44+
Problem:
45+
Albert is wondering how much pizza he can eat in one day. He buys 2 large pizzas and 2 small pizzas. A large pizza has 16 slices and a small pizza has 8 slices. If he eats it all, how many pieces does he eat that day?
46+
47+
Plan:
48+
First calculate the number of slices from the large pizzas. Then calculate the number of slices from the small pizzas. Finally add the number of slices from large and small pizzas.
49+
50+
- |
51+
Problem:
52+
Ken created a care package to send to his brother, who was away at boarding school. Ken placed a box on a scale, and then he poured into the box enough jelly beans to bring the weight to 2 pounds. Then, he added enough brownies to cause the weight to triple. Next, he added another 2 pounds of jelly beans. And finally, he added enough gummy worms to double the weight once again. What was the final weight of the box of goodies, in pounds?
53+
54+
Plan: First calculate the weight after adding the brownies. Then add the weight of the next batch of jelly beans. Finally double that amount.
55+
56+
- |
57+
Problem:
58+
Alexis is applying for a new job and bought a new set of business clothes to wear to the interview. She went to a department store with a budget of $200 and spent $30 on a button-up shirt, $46 on suit pants, $38 on a suit coat, $11 on socks, and $18 on a belt. She also purchased a pair of shoes, but lost the receipt for them. She has $16 left from her budget. How much did Alexis pay for the shoes?
59+
60+
Plan:
61+
First calculate how much Alexis spent except for shoes. Then calculate how much she spent in total by subtracting $16 from $200. Finally, subtract the amount spent except shoes from the total spent.
62+
63+
- |
64+
Problem:
65+
Jasper will serve charcuterie at his dinner party. He buys 2 pounds of cheddar cheese for $10, a pound of cream cheese that cost half the price of the cheddar cheese, and a pack of cold cuts that cost twice the price of the cheddar cheese. How much does he spend on the ingredients?
66+
67+
Plan:
68+
First calculate how much a pound of cream cheese costs. Then calculate how much the pack of cold cuts costs by multiplying that by 2. Finall add the cost of cheddar cheese, cream cheese, and cold cuts.
69+

‎examples/gsm8k/gsm8k-plan-few-shots.pdl‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
description: Grade School Math -- for every problem we generate a plan, then exectute and evaluate it.
2+
defs:
3+
problems:
4+
read: ./test.jsonl
5+
parser: jsonl
6+
7+
MAX_ITERATIONS: 50
8+
9+
planning:
10+
function:
11+
problem: str
12+
demos: [str]
13+
return:
14+
lastOf:
15+
- |
16+
Please generate a high-level plan for solving the following question.
17+
As the first step, just say what method and idea you will use to solve the question.
18+
You can reorganize the information in the question. Do not do the actual calculation.
19+
Keep your response concise and within 80 words.
20+
21+
- for:
22+
demo: ${ demos }
23+
repeat:
24+
${ demo }
25+
join:
26+
with: "\n"
27+
- text:
28+
- "\nProblem:\n"
29+
- ${ problem }
30+
- "\n"
31+
- model: ollama/granite3.2:8b
32+
33+
solve:
34+
function:
35+
plan: str
36+
return:
37+
text:
38+
- ${ plan }
39+
- |
40+
41+
The plan looks good! Now, use real numbers and do the calculation. Please solve the question
42+
step-by-step according to the high-level plan. Give me the final answer. Make your response short.
43+
- "\nThe answer is:\n"
44+
- model: ollama/granite3.2:8b
45+
46+
extract_final_answer:
47+
function:
48+
solution: str
49+
return:
50+
lastOf:
51+
- ${ solution }
52+
- Extract the result from the above solution into a JSON object with field "result" and a float as value. Remove any dollar signs or other symbols.
53+
- model: ollama/granite3.2:8b
54+
parser: json
55+
def: result
56+
spec: { "result": float }
57+
fallback:
58+
data:
59+
result: 0
60+
61+
compare_to_ground_truth:
62+
function:
63+
result: obj
64+
truth: str
65+
return:
66+
lastOf:
67+
- data: ${ truth }
68+
parser:
69+
regex: "(.|\n)*#### (?P<answer>([0-9])*)\n*"
70+
spec:
71+
answer: str
72+
def: ground_truth
73+
- if: ${ result.result|float == ground_truth.answer|float}
74+
then:
75+
1
76+
else:
77+
0
78+
79+
text:
80+
- defs:
81+
demos:
82+
read: demos.yaml
83+
parser: yaml
84+
for:
85+
problem: ${ problems }
86+
repeat:
87+
call: ${ planning }
88+
args:
89+
pdl_context: []
90+
problem: ${ problem.question }
91+
demos: ${ demos }
92+
max_iterations: ${ MAX_ITERATIONS }
93+
def: plans
94+
join:
95+
as: array
96+
97+
- for:
98+
plan: ${ plans }
99+
repeat:
100+
call: ${ solve }
101+
args:
102+
pdl_context: []
103+
plan: ${ plan }
104+
max_iterations: ${ MAX_ITERATIONS }
105+
def: solutions
106+
join:
107+
as: array
108+
109+
- for:
110+
solution: ${ solutions }
111+
repeat:
112+
call: ${ extract_final_answer }
113+
args:
114+
pdl_context: []
115+
solution: ${ solution }
116+
max_iterations: ${ MAX_ITERATIONS }
117+
def: results
118+
join:
119+
as: array
120+
121+
- for:
122+
result: ${ results }
123+
problem: ${ problems[:MAX_ITERATIONS] }
124+
repeat:
125+
call: ${ compare_to_ground_truth }
126+
args:
127+
pdl_context: []
128+
result: ${ result }
129+
truth: ${ problem.answer }
130+
max_iterations: ${ MAX_ITERATIONS }
131+
def: stats
132+
join:
133+
as: array
134+
135+
- "\nAccuracy: ${ stats|sum / MAX_ITERATIONS * 100}% "

0 commit comments

Comments
 (0)