SWEMentor – AI-Powered Code Analysis and Mentoring Tool

Video version

If you prefer the video version, you can watch it on YouTube:

Introduction

“What if I could build a tool that automatically reviews code and provides suggestions for improvement?”

That was my first thought when I was brainstorming weekend project ideas. I like to reflect on my own code and think about how I could have done something better. The idea seemed straightforward enough: build a system that could analyze code and provide helpful suggestions similar to what a senior developer might offer during a code review.

I decided to challenge myself further by building this entire project over a weekend using Cursor, an AI-powered development environment. I was curious to see how much I could accomplish in weekend afternoons and evenings with modern AI tooling. This wasn’t about creating a production-ready system; it was about validating the concept and seeing if automated code mentoring could provide genuine value.

The result is SWEMentor (abbreviated for Software Engineering Mentor), a code analysis and mentoring tool that can detect various code issues and suggest improvements.

Demo showcase

  • The main interface of SWEMentor – clean and straightforward, allowing users to select their project folder for analysis
    The main interface of SWEMentor – clean and straightforward, allowing users to select their project folder for analysis
  • The main interface of SWEMentor – clean and straightforward, allowing users to select their project folder for analysis
    The main interface of SWEMentor – clean and straightforward, allowing users to select their project folder for analysis
  • The system analyzing a Python project
    The system analyzing a Python project
  • SWEMentor showing detailed analysis of code issues alongside improvement suggestions
    SWEMentor showing detailed analysis of code issues alongside improvement suggestions
  • A detailed list of detected issues (can contain false positives)
    A detailed list of detected issues (can contain false positives)
  • Relevant patterns & suggestions for the detected issues
    Relevant patterns & suggestions for the detected issues

The code issues detailed inside the Detailed Analysis are actually legitimate! You can see it for yourself by visiting my Transformer from Scratch codebase. Note: It’s the same project as in the demo above, I just named the folder “Attention_is_all_you_need”. If I wanted to improve my Transformer from Scratch implementation, I could surely heed the advice of putting all of the constants in one file instead of having them re-defined in each file.

Architecture explanation

SWEMentor is a multi-agent RAG AI system. It is composed of different agents which are specialized in different tasks. The agents communicate with one another and their output is finally passed into an LLM, which provides the detailed analysis.

In particular, the architecture of SWEMentor consists of the following parts:

  • Code Analyzer agent: This agent analyzes code using traditional methods – an abstract syntax tree. This creates a list of issues which is then passed into the RAG agent.
  • RAG agent: This agent takes in the issues found by the Code Analyzer agent and it retrieves the most relevant entries from the code issues and code patterns collections. This allows for embedding custom software engineering knowledge into SWEMentor, making it more customizable.
  • LLM service: This service takes the output from the above two agents, puts it into a prompt and sends it over to an LLM. For the LLM I used Anthropic’s Claude 3.5 Sonnet (latest version as of the time of this writing).
  • Coordinator agent: An agent which coordinates all of the previously mentioned agents.

The entire system was built to be modular, allowing for easy addition of new analysis types or improvement of existing ones. The focus was on creating a working proof-of-concept that could demonstrate the value of automated code mentoring.

Tech stack

The tech stack consisted of the following:

  • For the Code Analyzer agent I used Python’s ast module.
  • For the RAG agent I used ChromaDB.
  • For the LLM, I used Anthropic’s Claude 3.5 Sonnet (latest version).
  • For the backend, I used FastAPI.
  • Frontend was created in HTML, CSS (Bootstrap) and JavaScript. Here I relied on Cursor, as I focused on the AI parts.

Future work

While the weekend prototype successfully demonstrates the concept, there are several areas for improvement:

  • Code Analyzer agent issue detection: The number of issues Code Analyzer agent detects could be expanded. This requires careful thought as the analysis is done deterministically (via an abstract syntax tree), but I believe it’s possible to make the analysis better.
  • RAG agent retrieval: In some cases, I’ve noticed duplicated retrievals and retrievals which weren’t so relevant. Improving retrieval would be beneficial.
  • More in-depth suggestions: While the current suggestions for code improvement make sense, in future iterations I’d like to build more in-depth suggestions. For example, there is a difference between implementing a scientific paper from scratch vs building a scaleable production-ready system and the code improvement suggestions should reflect that. Aditionally, I’d like SWEMentor to behave not only as a senior software engineer, but also as someone with subject matter expertise. What I mean by this is that it should tell me: “I see you implemented the masking mechanism in your Transformer in this way. It’s not bad, but here’s how it’s usually done: …”.
  • Dealing with large codebases: Some thought should be taken to make sure SWEMentor works on large codebases. One idea is to have SWEMentor analyze your code feature-by-feature and suggest improvements. If you wanted to analyze a large codebase, that would require some thoughful prompt engineering centered around how to best present this to the LLM (as the entire codebase doesn’t fit into the context window).
  • Multi-language support: The current implementation works only with Python. Adding support for JavaScript, TypeScript, and other popular languages would make the tool more versatile.

Conclusion

Building SWEMentor over a weekend was an interesting experiment in rapid prototyping with AI-assisted development. The project successfully demonstrates that automated code analysis and mentoring can provide valuable insights, even in a prototype form.

What struck me most was how quickly I could go from idea to working implementation using Cursor. While the tool certainly accelerated development, the real work was in designing the system architecture, selecting appropriate analysis patterns, and ensuring the suggestions were actually helpful.

This project reinforced my belief that the future of development involves leveraging AI tools to amplify our capabilities. The impressive part isn’t just using these tools – it’s knowing how to direct them effectively to solve real problems. In this case, the problem was providing accessible code review and mentoring, something that could help developers improve their code quality even when working solo or in small teams.

While SWEMentor is far from perfect, it serves as a proof-of-concept for what’s possible when combining traditional code analysis techniques with modern AI capabilities. The fact that it can identify real issues and provide actionable suggestions after just a weekend of development shows the potential of this approach.