Bug fixing for mindspace opening issues and improvements to the calculator tool with bitwise operations and base conversions. Includes discussion of LLM instruction following challenges.
Fix problem opening mindspaces
Previously, if the user selected a directory to open as a mindspace we'd start the close the current one before discovering the problem. This meant the mindspace tabs were closed. Now this won't happen.
Improving the calculator tool
One of the earliest tools I added was the calculator tool. It's very useful, but it couldn't do bitwise operations or base conversions - i.e. it wasn't a programmer's calculator, just a scientific one.
To solve this I used a Metaphor prompt:
Role:
You are a world class python programmer, highly skilled in accurate and performant software development. You
are going to assist me in making modifications to my application.
Context:
# Pull in the default Python coding rules for the project.
Include: metaphor/python-rules.m6r
# The project only uses external dependencies if there's no other choice. This makes it
# much easier for the AI to know what it's working with.
Context: Dependencies
Leverage standard library tools before custom solutions, unless specifically instructed, or
unless a depdency is already in use.
Context: Tool use
You have a number of tools at your disposal. Before using any of them, consider whether you need to use them or
if you already have the information you require and only use them if you don't have it, or are unsure.
Context: Existing code
The following files are used in my application - as you have them here you do not need to re-read them:
# Embed all the files that the AI will need to modify the code successfully. If necessary,
# describe what those files are used for.
Embed: src/ai_tool/*.py
Embed: src/ai_tool/tools/*.py
Action:
# Describe the details of the change required.
My application provides tools that are used by LLMs. I've given you the code for the tool infrastructure
and for 3 of the tools.
The calculator tool currently handles scientific calculations but I would like to extend this to cover
a programmer's calculator needs. This should handle base conversions and bitwise operators.
The current design is modelled on python syntax via the ast library, so it seems to make sense to use
the same approach for these new capabilities.
I don't want you to code anything yet - I want to understand ways we can do this
The design was pretty reasonable, but then we got into one of those places where the LLM suggested something weird. This is exactly where "vibe coders" will come unstuck:

I nudged the design to remove the slightly mad use of multiple outputs!

That got some reasonable code, but Claude was still overcomplicating things...

This one actually made sense. It was interesting to see Claude indulge its tendency to overcomplicate things, however. Each of these incremental complexities eventually leads to a very complex codebase.
Following instructions
A variant on the calculator problem was that some LLMs would try to use the caret ^
symbol for exponentiation. This was in spite of the tool description saying it needed to use **
for exponentiation.
After adding the new bitwise ops, the use of ^
was no longer safe because something like 23 ^ 3
would generate a result of 20
, not 12167
. The weird thing was that some LLMs still tried to use ^
, despite being told that that now did bitwise xor.
This required a very explicit addition to the tool description to fix it:
"- The ^ operator is for bitwise XOR and must not be used for exponentiation - use ** or pow() instead\n"