Conversation
… camera integration - Implemented index.html for the demo interface with message input, connection status, and event console. - Developed app.js for WebSocket connection management, message handling, and UI updates. - Created audio-player.js and audio-recorder.js for audio playback and recording using Audio Worklets. - Added PCM player and recorder processors to handle audio data efficiently. - Integrated camera functionality for capturing images and sending them through the WebSocket.
…dio play button functionality
…ive connection handling
Summary of ChangesHello @imrenagi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces foundational support for the Gemini Live API, enabling real-time, bidirectional streaming interactions with large language models within the ADK-Go framework. It establishes a new 'live mode' for agents, complete with a dedicated request queue for handling continuous input streams like audio and video. The changes extend core agent interfaces and configurations to support these new streaming capabilities, and a comprehensive example application is provided to showcase the end-to-end live streaming workflow, including a WebSocket server and client-side media handling. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces initial support for the Gemini Live API, enabling real-time, bidirectional streaming interactions. The changes are extensive, adding new RunLive methods to agents, a LiveRequestQueue for managing real-time inputs, and a comprehensive example application to demonstrate the new capabilities. The implementation looks promising and follows existing patterns in the codebase. My review focuses on potential issues such as a possible panic, areas for refactoring to reduce code duplication, and suggestions for improving API clarity and logging consistency.
…on options; update live connection handling for grounding metadata
…Context with new caching and session resumption methods
…on features; update InvocationContext and Runner for improved state management
… flow request handling.
…ve connection and add transcription accumulation logic.
…or live flows, and simplify live connection error handling.
…runner's event handling, and enhance live example with zerolog, artifact service, and transcription fields.
…ue`, and clarify `Channel` usage for timeout handling.
…ead of `fmt.Printf` for live connection handling.
…stantiation in the example.
…ToRun` signature, and pass session directly to invocation context.
…context creation.
|
hi @dpasiukevich , sorry for tagging you because im not sure who to tag. but, I would like to get your opinion and hopefully more person (please help to tag relevant team or person) on this PR. It is currently working for basic gemini live features. The missing things that is still working in progress are:
Thanks! |
…n events and integrate tool response handling in the live model flow.
…t a flush event return condition.
…e agent interactions.
…ling, concurrency, and resource management.
…ess` goroutine return an error channel and merging it in `Receive`.
… a centralized results channel, and unified lifecycle management to improve concurrency safety, error propagation, and resource cleanup.
…ils and delete the bugs documentation.
…ment and buffer underflow handling.
…ove unnecessary logging.
…nal handling and invocation context management for resumability.
…utable fields of `InvocationContext`.
…sages when resumability is enabled and filter empty LLM responses containing only GoAway.
…fine live session resumption update processing.
|
hi, any updates? i also need live api support |
Hi, i decided to give a try in implementing initial implementation for gemini live api support #550 in adk-go. This is a work in progress PR. I hope that I can get initial feedback from the community about few fundamental things like design, code changes, etc. I tried to see how it was done on adk-python and implement it in go.
Some highlights:
RunLivemethods in theAgentinterface andllmagent.LiveRequestQueueto manage real-time inputs (like audio and video) and extendedInvocationContextto support this queue, facilitating continuous interaction with live models.StreamingModeBidihas been added toagent.RunConfig, along with fields forResponseModalities,SpeechConfig, and audio transcription settings, to configure live connections.examples/live) that demonstrates the full live streaming workflow, featuring a WebSocket server, client-side audio/video handling, and a chat UI.Connectmethod andLiveConnectioninterface for the Gemini model, allowing it to establish and manage live bidirectional streams with the LLM.Notes: I havent really clean up some unnecessary changes like logger, docker compose, etc. Will move it later when finalizing this MR. Thanks!