Developer Agent for Slicer

As others, I have been using LLMs to help develop functionality in the Slicer. This is typically writing code in VS, then loading it into slicer and testing it, then copy and pasting the error back to the agent to iterate and fix the error.

I found this a bit intimidating (particularly VS code) as someone who doesn’t do programming as a day job. Particularly given nowadays our typical advise on the forum is to say “you can do this with chatcpt”, I thought if I can streamline a few things, and actually do all the work directly inside the Slicer. The idea is to do the prompting in Slicer (whether for a script or a module), let it run, and if it fails, capture the error and send it back to itself to correct it. Then the resultant script is loaded into the scene as a python text node and can be opened and edited inside our Script Editor (which I added a few more things that I think made it more usable, definitely for me).

At this point, as far as I can tell all of this is working. I am coming to a point that my technical skills and understanding is not sufficient to guide the internal prompting about how the agent should behave (as opposed to the user’s prompt about the task). I think it needs an insider view to edit those and create an agent behavior that’s more accurate in the first attempt than the iterating over endlessly. By the way this is all written via vibe coding, so I am not even claim the code is meaningful, but it does seem to do the job in the way I thought it might be useful. I am sure there are better ways to accomplish this, and probably use the tokens more sparingly.

If you want to give it a try, it is here GitHub - muratmaga/SlicerDeveloperAgent: an agent to develop Slicer modules through Gemini . You need a GH token to access the models and the Script Editor extension is a dependency. While there is no documentation, it is quite self-explanatory and looks something like this:

The goal is not to replace IDEs or people’s development environment, but to lower the barrier for people who don’t do programming for a living or would not even consider doing this since the prerequisites it. Maybe start with a simple script. Once that’s working perhaps try to convert it to a module that is more friendly. Or edit the existing python modules for additional tasks. I don’t think I can expand this any further on my own (due to time and said reasons), but if anyone wants to help out, please do so. This might be a nice PW activity.

1 Like