Closed
Description
I cannot figure out how to upload a base64 encoded PDF and use it as an input. I have changed the basic/local-image.ts code slightly, no luck. I'm beginning to think this is a bug. I have tested using a file id that has been uploaded to OpenAI file storage instead and that DOES work.
Error is:
requestID: 'req_540ac48f3181b045213000eea2d22ea9',
error: {
message: "Unknown parameter: 'input[0].content[0].detail'.",
type: 'invalid_request_error',
param: 'input[0].content[0].detail',
code: 'unknown_parameter'
},
I really just commented out these lines:
//type: 'input_image',
//image: `data:image/jpeg;base64,${b64Image}`,'
and replaced with:
type: 'input_file',
file: `data:application/pdf;base64,${b64Image}`,'
Full example code is below:
import fs from 'fs';
import path from 'path';
import { Agent, run } from '@openai/agents';
const bisonImagePath = path.join(__dirname, 'media/my-pdf.pdf');
function imageToBase64(imagePath: string): string {
const imageBuffer = fs.readFileSync(imagePath);
return imageBuffer.toString('base64');
}
async function main() {
const agent = new Agent({
name: 'Assistant',
instructions: 'You are a helpful assistant.',
});
const b64Image = imageToBase64(bisonImagePath);
const result = await run(agent, [
{
role: 'user',
content: [
{
type: 'input_file',
file: `data:application/pdf;base64,${b64Image}`,
//type: 'input_image',
//image: `data:image/jpeg;base64,${b64Image}`,
providerData: {
detail: 'auto',
},
},
],
},
{
role: 'user',
content: 'What do you see in this image?',
},
]);
console.log(result.finalOutput);
// This image shows a large American bison standing on a grassy hill. The bison has a shaggy brown coat, with parts of its fur shedding, and prominent curved horns. The background is mostly a light, overcast sky, which makes the bison stand out prominently in the image. There is green grass and some small wild plants in the foreground. The overall scene appears natural and serene, likely in a prairie or grassland environment.
}
if (require.main === module) {
main().catch(console.error);
}