• 7 min read
We have more than 50 active projects at RIGANTI, and most are developed using Azure DevOps, an all-in-one developer platform that hosts Git repositories and provides tools for work tracking, automated builds & releases, and more.
However, in recent years, many new AI-related features have been coming to GitHub, while Azure DevOps has seen few updates. We wanted these new features too, but migrating all the projects to GitHub would be challenging. We rely heavily on Azure Pipelines, Azure Boards, and other Azure DevOps features. GitHub offers replacements for some of them, but not all. We also wanted to avoid having the project scattered between multiple tools, for example, having a Git repository on GitHub while keeping work items in Azure DevOps.
That led us to building RIGANTI AI Coder - a tool that integrates with Azure DevOps and provides the features you know from GitHub Copilot. We started calling it Mr. RIGANTI, because it has been pretty awesome.
Code Review on Pull Requests
You can assign a pull request to Mr. RIGANTI. He'll look at all changes and post comments with his thoughts, including a summary comment with general recommendations. We've played with the prompt to achieve a high rate of useful comments and a low number of nitpicks. The better the pull request or the assigned issue is described, the more context our new colleague gets. He also respects the project-specific instructions in the agents.md file.
First, I assigned the pull request to Mr. RIGANTI in Azure DevOps:
A pipeline was run in the background, and within a few minutes, Mr. RIGANTI started posting comments with his observations:
Once the code review was complete, Mr. RIGANTI posted the final summary comment:
This is not a replacement for a proper human code review, which is crucial in all RIGANTI projects. However, it serves as the first round of feedback, and once the developers resolve the AI's findings, a human colleague will review the pull request.
If new changes are pushed to the branch, Mr. RIGANTI will rerun and post additional comments on the changes. Currently, he doesn't mark comments as resolved when the issues are fixed, but we'll definitely add this feature in the future.
Implement a Work Item
A substantial number of tasks on our projects are isolated and don't require overall project knowledge. We can create a Requirement, Task, or Bug in Azure DevOps and assign it to Mr. RIGANTI. Once it moves to the Active state, Mr. RIGANTI will create a new branch, draft a pull request, and start working on it. He updates the PR description and status to reflect the current progress. To provide him with more context, we also include information from parent work items in the prompt (Epic and User Story).
As you can see, I created a work item with a brief description of the issue. It is linked to a parent work item that describes the particular project area. As you can see, to improve the likelihood of success, I included the exact file names to ensure Mr. RIGANTI understands what I mean. Since he cannot run the application and "see" the pages, it might be challenging for him to guess the correct location from my vague description:
Mr. RIGANTI immediately posted a comment with a link to the pipeline run, indicating that he had started working. A Git branch and a draft pull request have been created. In a few minutes, the following changes were implemented:
Respond to Comments
We've also implemented comment reactions. If you comment on Mr. RIGANTI'S pull request, mention him, or respond to his comment, he'll try to implement the requested change. Most projects are difficult to run without external dependencies, such as databases or APIs; therefore, Mr. RIGANTI can only attempt to compile the project and can easily miss something that needs to be done. However, it is quite easy for us to checkout the branch locally, run the application, and give him additional feedback. If you just copy-paste the error message and give it to him, he'll usually figure out what's wrong and fix it.
I ran the project locally and noticed some issues. As you can see, I didn't polish my feedback much - basically, I just pasted the errors I saw on the screen:
Mr. RIGANTI posted another comment stating that he had started working, and within 2 minutes, he updated it to indicate success. He also posted another comment that summarizes his changes.
How It Works
The first version we had in the autumn was a simple PowerShell script that launched Codex CLI in the directory where the repository was cloned. It could only perform the code review; it was essentially a long prompt with instructions on how to call the Azure DevOps API to post comments on the PR. It was quite error-prone, but we wanted to see whether the quality of code review would be sufficient and whether it makes sense to continue in this approach.
Once it was clear that the tool was useful, we implemented a more robust solution that can be operated with a reasonable level of security. Because the AI evolves rapidly and we didn't want to rely on a single provider, the entire solution was designed to be extensible.
Currently, Mr. RIGANTI uses Codex CLI for all work, but it will be easy to switch to another tool, such as GitHub Copilot CLI or OpenCode. Everything runs on build agents as part of an Azure DevOps build pipeline, where it builds the project and verifies there are no compile errors. We'd like to reach a state where Mr. RIGANTI can run the application and use the Playwright MCP server to interact with the app, but this will be a longer journey.
The RIGANTI AI Coder consists of two parts:
- A web interface where you can authenticate to your Azure DevOps instance and roll out Mr. RIGANTI to your projects. Basically, it installs a few webhooks and creates a build pipeline in which Mr. RIGANTI can run. This interface also handles Azure DevOps webhooks and orchestrates the pipeline runs.
- A console agent application that is launched in the build pipeline in the context of your repository. It obtains the prompt, ensures the last version of Codex CLI (or other configured coding tool) is installed, passes it the configuration and API keys based on project settings, and gives it an MCP server to be able to interact with Azure DevOps (for example, post a comment or update the current PR description).
To run this, we need an additional user account in Azure DevOps (for Mr. RIGANTI) and an OpenAI API key (or an API key from another provider supported by the Codex CLI).
We are still determining which tasks are good to assign to Mr. RIGANTI and where he is likely to fail. It depends on the project context and the required depth of understanding of the problem domain. So far, our experience is that it usually doesn't do 100% of the task, but it helps us to be much faster. Instead of spending 2 hours on a pull request to manually make all the code changes, I usually need only 20-40 minutes to review and finalize the work Mr. RIGANTI has done.
If you want a similar experience in your Azure DevOps, contact us - Mr. RIGANTI will be happy to help!