cognify.

Applied R&D · AI · Data · Analytics

Deterministic mathematical capabilities in Qwen 3.5 0.8B

← Back to logbook

We are experimentally developing a local agent based on Qwen 3.5 0.8B, running via Ollama. The goal is to explore an architecture in which a small language model does not have to solve all the operations necessary to answer a query by itself, but rather can rely on specialized external skills.

The basic idea is to separate two functions. Qwen interprets what the user is requesting, decides what it needs to do, and can construct a response in natural language. The skills, on the other hand, perform specific operations using external software or deterministic functions.

The general scheme is:

text
Usuario
   ↓
Qwen 3.5 0.8B
   ↓
¿Puede/necesita utilizar una capacidad externa?
   ↓
Skill
   ↓
Resultado
   ↓
Qwen
   ↓
Respuesta / nueva decisión

An important feature of the agent is that this process can be repeated. After receiving the result of a skill, Qwen re-evaluates the task and may decide to use another tool. This allows for experimentation not only with tool use, but with the sequential composition of capabilities.

At this stage, we have incorporated a mathematical skill.


Why a mathematical skill?

Language models can produce mathematical results directly, but calculation is part of the model's generation. Our interest was not to improve Qwen through more instructions for calculating, but to allow it to delegate mathematical operations to deterministic functions.

Initially, we created three tools:

text
raiz_cuadrada(numero)
potencia(base, exponente)
factorial(numero)

These functions use the Python math library.

The first version of math_skill.py was conceptually:

python
import math

def raiz_cuadrada(numero):
    return math.sqrt(numero)

def potencia(base, exponente):
    return math.pow(base, exponente)

def factorial(numero):
    return math.factorial(numero)

Then we registered these functions as tools available to Qwen.

This quickly produced an interesting behavior.

When we asked:

text
¿Cuánto es 2+2?

Qwen recognized that it was a mathematical operation, but it did not have a tool for addition. Among the existing mathematical tools, it found potencia (power) and called:

text
potencia(base=2, exponente=2)

The result was 4.

The numerical response accidentally matched 2+2, but the procedure was incorrect: Qwen had substituted an addition with a power simply because both operations produced the same value in that case.

The test allowed us to distinguish a problem that was not one of calculation but of capability selection.

Therefore, we added an explicit rule to the agent:

text
No uses una herramienta para una operación diferente,
aunque accidentalmente pueda producir el mismo resultado.

But we also noticed a problem in our own design: we were asking the model to outsource mathematical calculations without providing it a tool for the most common arithmetic operations.


Incorporating a calculator

We then decided to expand math_skill.py with a function:

text
calcular(expresion)

Its goal is to solve expressions involving:

text
+
-
*
/
//
%
**
()

We do not use eval() directly. The calculator parses the expression via ast and accepts only specific nodes and mathematical operators.

The current implementation of the skill is:

python
import math
import ast
import operator


# =========================================================
# SKILL — MATEMÁTICA
# =========================================================


# =========================================================
# CALCULADORA
# =========================================================

OPERACIONES = {
    ast.Add: operator.add,
    ast.Sub: operator.sub,
    ast.Mult: operator.mul,
    ast.Div: operator.truediv,
    ast.FloorDiv: operator.floordiv,
    ast.Mod: operator.mod,
    ast.Pow: operator.pow,
    ast.USub: operator.neg,
    ast.UAdd: operator.pos
}


def calcular(expresion: str) -> float:
    """
    Calcula una expresión matemática.

    Permite suma, resta, multiplicación, división,
    potencias, módulo y paréntesis.
    """

    arbol = ast.parse(
        expresion,
        mode="eval"
    )

    def evaluar(nodo):

        if isinstance(nodo, ast.Constant):

            if isinstance(nodo.value, (int, float)):
                return nodo.value

            raise ValueError(
                "Solo se permiten números."
            )

        if isinstance(nodo, ast.BinOp):

            tipo_operacion = type(nodo.op)

            if tipo_operacion not in OPERACIONES:
                raise ValueError(
                    "Operación matemática no permitida."
                )

            izquierda = evaluar(nodo.left)
            derecha = evaluar(nodo.right)

            return OPERACIONES[tipo_operacion](
                izquierda,
                derecha
            )

        if isinstance(nodo, ast.UnaryOp):

            tipo_operacion = type(nodo.op)

            if tipo_operacion not in OPERACIONES:
                raise ValueError(
                    "Operación matemática no permitida."
                )

            valor = evaluar(nodo.operand)

            return OPERACIONES[tipo_operacion](
                valor
            )

        raise ValueError(
            "La expresión contiene elementos no permitidos."
        )

    return evaluar(arbol.body)


# =========================================================
# RAÍZ CUADRADA
# =========================================================

def raiz_cuadrada(numero: float) -> float:

    if numero < 0:
        raise ValueError(
            "No se puede calcular la raíz cuadrada real "
            "de un número negativo."
        )

    return math.sqrt(numero)


# =========================================================
# POTENCIA
# =========================================================

def potencia(base: float, exponente: float) -> float:

    return math.pow(base, exponente)


# =========================================================
# FACTORIAL
# =========================================================

def factorial(numero: int) -> int:

    if numero < 0:
        raise ValueError(
            "El factorial requiere un número no negativo."
        )

    return math.factorial(numero)

Qwen then had four mathematical capabilities available:

text
calcular
raiz_cuadrada
potencia
factorial

We also made their functions explicit in the prompt:

text
- Usa calcular para operaciones aritméticas como suma, resta,
  multiplicación, división y combinaciones de estas operaciones.

- Usa raiz_cuadrada únicamente para calcular raíces cuadradas.

- Usa potencia únicamente para calcular potencias.

- Usa factorial únicamente para calcular factoriales.

- No uses una herramienta para una operación diferente, aunque
  accidentalmente pueda producir el mismo resultado.

First test: calculator selection

We asked:

text
Tú: cuanto es 2+3

This time Qwen correctly identified the available capability. In its monologue, it indicated:

text
El usuario está pidiendo calcular la expresión matemática "2+3".
Esta es una operación aritmética que puedo resolver directamente
usando la herramienta de cálculo disponible en el sistema.

La herramienta `calcular` permite realizar operaciones matemáticas
como suma, resta, multiplicación, división, potencias y paréntesis.

Then it called:

text
Skill: calcular
Argumentos: {'expresion': '2+3'}
Resultado: 5

and responded:

text
El resultado de 2+3 es 5.

The same test was performed with a multiplication:

text
Tú: cuanto es 286*29

Qwen executed:

text
Skill: calcular
Argumentos: {'expresion': '286*29'}
Resultado: 8294

The incorporation of the calculator had solved the previously observed problem: Qwen now had an appropriate capability for general arithmetic operations and began selecting it correctly.


Second test: combining operations

The next objective was more important.

We wanted to see if Qwen could combine mathematical capabilities, using the result produced by one tool as input for another.

We proposed:

text
√(28 / 3)

Solving it using our skills requires conceptually:

text
calcular("28 / 3")
        ↓
9.333333333333334
        ↓
raiz_cuadrada(9.333333333333334)
        ↓
3.0550504633038935

We did not specifically program this sequence. The agent only knows the available tools and can decide again what to do after each result.


First attempt: incorrect argument

Given:

text
Tú: cuanto es la raiz cuadrada de 28/3

Qwen understood that it had to calculate:

text
√(28/3)

but tried to do:

text
Skill: raiz_cuadrada
Argumentos: {'numero': '28/3'}

The call failed:

text
ERROR: '<' not supported between instances of 'str' and 'int'

The reason was simple: raiz_cuadrada expects to receive a number, but Qwen had sent "28/3" as a text string.

What was interesting appeared after the error.

Qwen recognized:

text
La función `raiz_cuadrada` espera un número,
pero 28/3 es una expresión matemática.
Necesito convertirlo primero.

That is to say, it correctly identified that there was an intermediate result that it had to obtain before using raiz_cuadrada.

However, it did not manage to turn that conclusion into a new call to calcular. It entered a repetition of the same reasoning until it exhausted the generation.

The test showed that verbally recognizing the correct procedure does not guarantee executing it correctly.


Second attempt: incorrect interpretation of composition

We reformulated the query:

text
raíz cuadrada de la división de 28 en 3

This time Qwen interpreted that we were requesting two different results:

text
1. raíz cuadrada de 28
2. división de 28 por 3

and executed both tools separately:

text
raiz_cuadrada(28)
→ 5.291502622129181

calcular("28 / 3")
→ 9.333333333333334

The tools worked correctly, but the interpretation of the relationship between them was incorrect.

This showed another distinction: it is one thing to correctly select a tool and another to understand the dependency structure between multiple operations.


Third attempt: successful composition

Finally, we made the structure explicit using parentheses:

text
raíz cuadrada de (la división de 28 en 3)

This time Qwen's monologue showed the correct decomposition:

text
Esto significa que debo:

1. Primero calcular 28 ÷ 3
2. Luego calcular la raíz cuadrada del resultado

In the first round, it executed:

text
Skill: calcular
Argumentos: {'expresion': '28 / 3'}
Resultado: 9.333333333333334

After receiving the result, Qwen reasoned again:

text
Bien, la división de 28 por 3 es 9.333333333333334.

Ahora necesito calcular la raíz cuadrada de este resultado:
√(9.333333333333334)

In the second round, it called:

text
Skill: raiz_cuadrada
Argumentos: {'numero': 9.333333333333334}
Resultado: 3.0550504633038935

In the third round, it stopped requesting tools and responded to the user.

For the first time, we observed a complete sequence:

text
Usuario
   ↓
Qwen interpreta el problema
   ↓
calcular("28 / 3")
   ↓
9.333333333333334
   ↓
Qwen recibe el resultado
   ↓
decide que necesita otra capacidad
   ↓
raiz_cuadrada(9.333333333333334)
   ↓
3.0550504633038935
   ↓
Qwen
   ↓
respuesta final

What we learned

The mathematical skill started as a simple way to provide deterministic operations to a small model, but the experiment ended up showing a more interesting problem.

There are at least three different levels:

text
1. EJECUCIÓN
   Una herramienta realiza correctamente una operación.

2. SELECCIÓN
   Qwen determina qué herramienta corresponde al problema.

3. COMPOSICIÓN
   Qwen determina qué herramientas necesita,
   en qué orden utilizarlas y cómo pasar el resultado
   de una como entrada de la siguiente.

The first level is deterministic: calcular("28/3") always performs the operation defined by the program.

The other two still depend on the model.

Qwen can select the wrong tool, it can construct arguments incompatible with the function, or it can misunderstand the relationship between multiple operations. It can even correctly describe what it should do and yet not execute it.

But the final test also showed that a model with 0.8B parameters can, under certain conditions, perform a sequential composition of external capabilities:

text
calcular
   ↓
resultado
   ↓
razonamiento
   ↓
raiz_cuadrada
   ↓
resultado

Therefore, the experiment did not just consist of “giving Qwen a calculator.” The mathematical skill became a first controlled test to study how a small model selects, combines, and sequences deterministic capabilities to solve a task that requires multiple steps.