Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 16184ba

Browse files
authored
gsm8k examples and results (#322)
* gsm8k code and results
1 parent b946730 commit 16184ba

18 files changed

+2949
-2
lines changed

examples/gsm8k/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
To run the programs in this file, you need the gsm8k benchmark available at:
2+
https://github.com/openai/grade-school-math

examples/gsm8k/math.pdl

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
defs:
2+
RITS_API_KEY:
3+
lang: python
4+
code: |
5+
import os
6+
result = os.environ["RITS_API_KEY"]
7+
solve:
8+
function:
9+
question: str
10+
return:
11+
lastOf:
12+
- "${ question }\n"
13+
- model: openai/ibm-granite/granite-3.1-8b-instruct
14+
def: solution
15+
parameters:
16+
api_key: ${RITS_API_KEY}
17+
api_base: https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/granite-3-1-8b-instruct/v1/
18+
extra_headers:
19+
RITS_API_KEY: ${RITS_API_KEY}
20+
- "\nwhat is the final answer from the above solution?\n"
21+
- model: openai/ibm-granite/granite-3.1-8b-instruct
22+
def: answer
23+
spec: {result: float}
24+
parser: json
25+
parameters:
26+
api_key: ${RITS_API_KEY}
27+
api_base: https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/granite-3-1-8b-instruct/v1/
28+
extra_headers:
29+
RITS_API_KEY: ${RITS_API_KEY}
30+
- data:
31+
solution: ${ solution }
32+
answer: ${ answer }
33+
truth:
34+
function:
35+
answer: str
36+
return:
37+
lang: python
38+
code:
39+
|
40+
result = float(answer.split("####")[-1].strip().replace("$", "").replace(",", ""))
41+
compare:
42+
function:
43+
ans1: float
44+
ans2: float
45+
return:
46+
lang: python
47+
code:
48+
|
49+
result = float(ans1) == float(ans2)
50+
51+
52+
text:
53+
- read: ../../../grade-school-math/grade_school_math/data/test.jsonl
54+
parser: jsonl
55+
contribute: []
56+
def: mathdata
57+
- for:
58+
problem: ${ mathdata }
59+
repeat:
60+
lastOf:
61+
- call: ${ solve }
62+
args:
63+
question: ${ problem.question }
64+
pdl_context: []
65+
def: solution
66+
- def: actual
67+
call: ${ truth }
68+
args:
69+
answer: ${ problem.answer }
70+
- def: compare
71+
lang: python
72+
code:
73+
|
74+
result = float(solution["answer"]["result"]) == actual
75+
- if: ${ compare }
76+
then:
77+
data:
78+
solution: ${ solution }
79+
matches: true
80+
else:
81+
data:
82+
solution: ${ solution }
83+
matches: false
84+
join:
85+
with: "\n"
86+

examples/gsm8k/math_no_sd.pdl

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
defs:
2+
RITS_API_KEY:
3+
lang: python
4+
code: |
5+
import os
6+
result = os.environ["RITS_API_KEY"]
7+
solve:
8+
function:
9+
question: str
10+
return:
11+
lastOf:
12+
- "${ question }\n"
13+
- model: openai/ibm-granite/granite-3.1-8b-instruct
14+
def: solution
15+
parameters:
16+
api_key: ${RITS_API_KEY}
17+
api_base: https://inference-3scale-apicast-production.apps.rits.fmaas.res.ibm.com/granite-3-1-8b-instruct/v1/
18+
extra_headers:
19+
RITS_API_KEY: ${RITS_API_KEY}
20+
- def: answer
21+
lang: python
22+
code: |
23+
try:
24+
result = float(solution.split("The answer is:")[-1].strip())
25+
except:
26+
result = 0
27+
- data:
28+
solution: ${ solution }
29+
answer:
30+
result: ${ answer }
31+
truth:
32+
function:
33+
answer: str
34+
return:
35+
lang: python
36+
code:
37+
|
38+
result = float(answer.split("####")[-1].strip().replace("$", "").replace(",", ""))
39+
compare:
40+
function:
41+
ans1: float
42+
ans2: float
43+
return:
44+
lang: python
45+
code:
46+
|
47+
result = float(ans1) == float(ans2)
48+
49+
50+
text:
51+
- read: ../../../grade-school-math/grade_school_math/data/test.jsonl
52+
parser: jsonl
53+
contribute: []
54+
def: mathdata
55+
- for:
56+
problem: ${ mathdata }
57+
repeat:
58+
lastOf:
59+
- call: ${ solve }
60+
args:
61+
question: ${ problem.question }
62+
pdl_context: []
63+
def: solution
64+
- def: actual
65+
call: ${ truth }
66+
args:
67+
answer: ${ problem.answer }
68+
- def: compare
69+
lang: python
70+
contribute: []
71+
code:
72+
|
73+
result = float(solution["answer"]["result"]) == actual
74+
- if: ${ compare }
75+
then:
76+
data:
77+
solution: ${ solution }
78+
matches: true
79+
else:
80+
data:
81+
solution: ${ solution }
82+
matches: false
83+
join:
84+
with: "\n"
85+

examples/gsm8k/math_no_sd_out.jsonl

Lines changed: 1319 additions & 0 deletions
Large diffs are not rendered by default.

examples/gsm8k/math_out.jsonl

Lines changed: 1321 additions & 0 deletions
Large diffs are not rendered by default.

src/pdl/pdl_dumper.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -282,7 +282,7 @@ def as_json(value: Any) -> JsonType:
282282
def parser_to_dict(parser: ParserType) -> str | dict[str, Any]:
283283
p: str | dict[str, Any]
284284
match parser:
285-
case "json" | "yaml":
285+
case "json" | "yaml" | "jsonl":
286286
p = parser
287287
case RegexParser():
288288
p = parser.model_dump()
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
What is APR?APR stands for Annual Percentage Rate. It is the annual interest rate charged for borrowing or earned through an investment, and it reflects the cost of a loan on an annual basis, including fees and compounding interest.yes
2+
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
What is APR?
2+
APR stands for Annual Percentage Rate. It is the annual interest rate charged for borrowing or earned through an investment, and it represents the actual yearly cost of funds over the term of a loan. It includes any fees or additional costs associated with the transaction.french
3+
4+
Translate the above to french
5+
"APR" signifie "Taux Annuel Équivalent" en français. Il est le taux d'intérêt annuel chargé pour emprunter ou gagné à travers une investissement, et il représente le coût réel annuer des fonds sur la durée d'un prêt. Il inclut toutes les frais ou coûts supplémentaires associés à la transaction.stop
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
Here is a simple Python function to compute the Fibonacci sequence:
2+
3+
```python
4+
def fibonacci(n):
5+
if n <= 0:
6+
return "Input should be a positive integer."
7+
elif n == 1:
8+
return 0
9+
elif n == 2:
10+
return 1
11+
else:
12+
a, b = 0, 1
13+
for _ in range(n - 2):
14+
a, b = b, a + b
15+
return b
16+
```
17+
18+
This function takes an integer `n` as input and returns the `n`th number in the Fibonacci sequence. The Fibonacci sequence is a series of numbers in which each number is the sum of the two preceding ones, usually starting with 0 and 1.
19+
Find a random number between 1 and 20
20+
15
21+
Now computing fibonacci(15)
22+
23+
def fibonacci(n):
24+
if n <= 0:
25+
return "Input should be a positive integer."
26+
elif n == 1:
27+
return 0
28+
elif n == 2:
29+
return 1
30+
else:
31+
a, b = 0, 1
32+
for _ in range(n - 2):
33+
a, b = b, a + b
34+
return b
35+
The result is: 377
36+
37+
Explain what the above code does and what the result means
38+
39+
The provided code is a Python function named `fibonacci(n)` that calculates the `n`th number in the Fibonacci sequence. The Fibonacci sequence is a series of numbers in which each number is the sum of the two preceding ones, usually starting with 0 and 1.
40+
41+
The function takes an integer `n` as input and returns the `n`th number in the Fibonacci sequence. If the input is not a positive integer, the function returns an error message.
42+
43+
In this case, the function is called with the argument `15`, which means it will calculate the 15th number in the Fibonacci sequence. The result of this calculation is `377`.
44+
45+
So, the number 377 is the 15th number in the Fibonacci sequence.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Hello
2+
Hello
3+
Did you say Hello?
4+
Yes, I did say Hello

0 commit comments

Comments
 (0)