Gemini in IDX: Enhancing Your Development Workflow with AI
Project IDX is enhanced by Gemini, Google's latest and most capable family of AI models. Let's explore how this integration can streamline your coding experience and give you a behind-the-scenes look at how your code interacts with Gemini in IDX.
Under the Hood: How the Gemini model works within IDX
Each IDX workspace runs on a dedicated virtual machine (VM) that is preconfigured to Gemini models via task specific APIs. Here's how it works:
1. Local Codebase Indexing: When you choose to enable it, a local code-indexer on your VM creates a searchable index of your codebase. This index stays on your machine. You can control which parts of your codebase are not indexed using a .aiexclude file, ideal for sensitive or proprietary code.
2. Tailored, Context-Aware Prompts: A prompt engine analyzes your current project, tools, errors, open files, and relevant code snippets (if indexing is enabled). This information is used to craft highly tailored prompts for Gemini in IDX, ensuring you receive the most relevant AI assistance.
3. Model Selection: IDX intelligently chooses the right Gemini model for the task: a small model for fast code completions, a larger streaming model for chat, and a powerful model for inline code generation and transformations.
4. Contextual Post-Processing: Gemini's responses are formatted for easy integration into your workflow. Code suggestions appear inline, explanations are displayed in the IDX AI chat, and tool actions require your approval.
Unleashing the Power of Codebase Indexing
To get the most out of IDX's AI features, enable codebase indexing. This allows Gemini to understand your project deeply and provide more relevant suggestions and answers.
To Enable: Go to your settings (.vscode/settings.json) and set "IDX.aI.enableCodebaseIndexing": true. You can also control this from the Gemini Chat window and click on the settings icon.
Important Note: When you enable codebase indexing and use Gemini in IDX features, relevant code snippets are sent to a Gemini model to provide contextually relevant responses. You can control which parts of your codebase are excluded in the codebase index by using a .aiexclude file, ideal for sensitive or proprietary code you don’t want to be used automatically as context for the various AI surfaces (e.g. Code Completion, Chat, and Inline Generation).
AI Assistance in Your Coding Workflow: Code Completion, Chat, and Generation
Code Completion
As you type, Gemini suggests the next piece of code based on your current file and, if enabled, relevant snippets from your project. Press Tab to accept the suggestion or use Cmd/Ctrl+left arrow to accept a word.
A smaller, faster model drives code completion in IDX. As you type, a local context/prompt engine analyzes your environment, considering factors like prefix, suffix, open files, and error context. This information is used to create and send a formatted request to the model. The model's response is then assessed for quality, and any responses below a certain threshold are filtered out.
Additionally, if the code completion API detects that the response contains any form of recitation, it includes a comment in the code along with a link to the AI License Log panel, displaying any licenses that the code might be subject to.
Pro tips:
Add comments to guide code completion: Code completion works even better if you start typing right after a code comment. In the example below, the comment hints to Gemini what the intended code should do.
Accept partial completions with Ctrl/Cmd + Right-Arrow: You don’t have to accept the entire completion — you can accept part of the completion by pressing (Ctrl + Right-Arrow, or Cmd + Right-arrow on a Mac). Press Escape at any time to dismiss the autocompletion.
Review license information for suggested code: If the code completion API detects that the suggested code contains any form of recitation and is subjected to licensed code, a reference is included as a comment in the code along with a link to the AI License Log panel. You can click on the link shown in code or you can access it from the Chat window toolbar.
Code Chat
Click the "Gemini" button or use Ctrl+Shift+Space (Cmd+Shift+Space on Mac) to open the chat panel. Gemini in IDX can help answer questions about your code, explain concepts, and even modify your code with your approval.
Gemini in IDX’s code chat is powered by a larger model specifically fine-tuned for coding related tasks. The prompt engine intelligently selects the appropriate context for each user query. If the model decides to take action, it invokes the action handler, which completes the action after receiving user permission. Actions can be repeated until the model determines your query has been adequately addressed. A dedicated streaming response handler manages code formatting, its corresponding UI, and Recitation.
Pro tips:
Ask IDX-specific questions with the /idx command: You can use the “/idx” slash command to ask questions related to Project IDX itself. For example, if you want to know how to install python in IDX (which uses Nix to set up its environment), you can ask “/idx how do I install Python?”. And if you are importing your own project via Git, and want web previews setup, just ask “/idx how do I setup web preview for this project?”.
Ask about your current file: Gemini in IDX can use the currently open file for context so you can easily ask questions about the current file you’re in. For example, you can ask to change the current file, an express.js server, to add a path.
Leverage Gemini's understanding of your entire project (if indexing is enabled): Gemini in IDX can use your codebase index & contextual understanding when answering your questions. So you can simply ask, “In my current file, how do I change the callServer() method to use fetch instead of XMLHttpRequest?”.
Note: Make sure codebase indexing is enabled as discussed above.
Create multiple threads for different conversations: You can create multiple chat threads with Gemini in IDX. Keep conversations short and to a single context to avoid the chat history getting in the way of AI responses when a new request is totally different from the current conversation.
Easily insert code from chat into your editor: Use the “Insert code” button on any generated code to move it to the current cursor location on the editor.
Use right-click shortcuts for common tasks like explaining code:
We’ve added context-menu items for common repetitive tasks for quick easy access. Just right click, and ask Gemini in IDX to do common things like “Explain code” or “Add comments”.
Inline Code Generation
Press Cmd/Ctrl + I to trigger inline code generation. Ask Gemini in IDX to write or edit code directly in your editor.
Code generation is driven by a powerful non-streaming endpoint specifically optimized for coding tasks. The prompt engine intelligently selects the appropriate context & prompt for each user query. The response handler takes different actions depending on the model's output, such as inserting new code into the editor, modifying existing selected code, displaying code explanations in the chat if the response isn't intended for the codebase, or handling recitation information.
Pro tips:
Use slash commands like /fix, /errorHelp, /addComments, /explain, and /transform for quick coding assistance: You can use slash commands like “/fix” to allow Gemini in IDX to automatically fix errors for you. Some commands like “/fix” will also get the error context from the editor and intuitively provide better recommendations.
Experimental Feature: Interactive Chat Threads
Early testers can explore our new interactive chat feature, where Gemini in IDX can answer questions, execute commands, run tests, and even make code changes with your permission.
For example, if you ask Gemini to run the unit tests for you, it will attempt to run the command required, such as ”npm run test”, instead of just giving you information on how to do it. Gemini in IDX can also take multiple actions on your behalf until you are satisfied with the outcome. And of course, all this happens with your permission – you can always guide Gemini in IDX by declining, accepting or amending its suggestions.
Stay tuned for more information on how you can sign up to be an early tester.
Help Us Improve!
Your feedback is invaluable. Use the thumbs up/down buttons to rate AI responses and help Gemini in IDX learn and improve.
Stay Tuned for More!
We’re constantly working to improve, enhance and update Gemini in IDX to further simplify and streamline your developer workflow. We’re also always working to address your feature requests and bugs you flag for us – so thank you for your continued support and engagement! Please keep the feedback coming by filing bugs and suggesting / voting on feature requests.
We have lots more in store for you, so be sure to keep an eye out for the latest from Project IDX by engaging with us on the new community forums and checking out our blog for product announcements and updates from the team.
For walkthroughs and more information on the features mentioned above, check out our documentation page. If you haven’t already, visit our website to try out Project IDX and join us on our journey.
We can’t wait to see what you create with Project IDX!