cognify.

Applied R&D · AI · Data · Analytics

Qwen 3.5 0.8B + Wikipedia: Incorporating External Knowledge Access

← Back to logbook

Objective. Explore whether Qwen 3.5 0.8B can incorporate a Wikipedia access skill and autonomously combine it with other system capabilities.

1. Incorporating wiki_skill

wiki_skill.py was created as a new external agent capability. The skill uses the Spanish Wikipedia API and exposes two distinct operations:

  • buscar_wikipedia: performs a search and returns candidate article titles.
  • leer_wikipedia: retrieves the introductory text of a selected article.

This separation was deliberate: the model does not receive the response directly from a query, but may have to execute a search → select → read sequence.

The implementation used was:

python
import requests

WIKIPEDIA_API = "https://es.wikipedia.org/w/api.php"
HEADERS = {"User-Agent": "QwenExperimentalAgent/0.1 (local research project)"}

def buscar_wikipedia(consulta: str, limite: int = 3) -> str:
    params = {
        "action": "query",
        "list": "search",
        "srsearch": consulta,
        "format": "json",
        "utf8": 1,
        "srlimit": limite,
    }
    try:
        response = requests.get(WIKIPEDIA_API, params=params, headers=HEADERS, timeout=10)
        response.raise_for_status()
        data = response.json()
        resultados = data["query"]["search"]
        if not resultados:
            return "No se encontraron resultados en Wikipedia."
        titulos = [r["title"] for r in resultados]
        return "\n".join(titulos)
    except requests.RequestException as e:
        return f"ERROR_WIKIPEDIA: {e}"

def leer_wikipedia(titulo: str) -> str:
    params = {
        "action": "query",
        "prop": "extracts",
        "exintro": True,
        "explaintext": True,
        "redirects": True,
        "titles": titulo,
        "format": "json",
        "utf8": 1,
    }
    try:
        response = requests.get(WIKIPEDIA_API, params=params, headers=HEADERS, timeout=10)
        response.raise_for_status()
        data = response.json()
        pages = data["query"]["pages"]
        pagina = next(iter(pages.values()))
        if "missing" in pagina:
            return "ERROR_WIKIPEDIA: El artículo no existe."
        extracto = pagina.get("extract", "")
        if not extracto:
            return "ERROR_WIKIPEDIA: El artículo no contiene un resumen."
        return extracto
    except requests.RequestException as e:
        return f"ERROR_WIKIPEDIA: {e}"

The first implementation produced an HTTP 403 Forbidden error. The query began working correctly after adding a User-Agent to the requests.

The isolated test of the skill returned:

Marie Curie
Universidad Pierre y Marie Curie
Hospital Municipal de Oncología Marie Curie

This allowed us to separate the functioning of the Wikipedia infrastructure from the model's subsequent behavior.

2. Integrating Wikipedia into the Agent

The functions were incorporated into the same toolset as the linguistic and mathematical capabilities:

python
from skills import contar_letra, contar_letras, revisar_ortografia
from math_skill import calcular, raiz_cuadrada, potencia, factorial
from wiki_skill import buscar_wikipedia, leer_wikipedia

TOOLS = {
    "contar_letra": contar_letra,
    "contar_letras": contar_letras,
    "revisar_ortografia": revisar_ortografia,
    "calcular": calcular,
    "raiz_cuadrada": raiz_cuadrada,
    "potencia": potencia,
    "factorial": factorial,
    "buscar_wikipedia": buscar_wikipedia,
    "leer_wikipedia": leer_wikipedia
}

The agent provides Qwen with all available tools in every round:

python
respuesta = chat(
    model=MODELO,
    messages=messages,
    tools=list(TOOLS.values()),
    options={"temperature": 0}
)

When Qwen requests a tool, the agent identifies its name and arguments:

python
for llamada in llamadas:
    nombre = llamada.function.name
    argumentos = llamada.function.arguments

    if nombre not in TOOLS:
        resultado = "ERROR: herramienta desconocida"
    else:
        try:
            resultado = TOOLS[nombre](**argumentos)
        except Exception as e:
            resultado = f"ERROR: {e}"

Finally, the result of the skill returns to the model's context:

python
messages.append({
    "role": "tool",
    "tool_name": nombre,
    "content": str(resultado)
})

Therefore, the agent does not contain a programmed sequence of the Wikipedia → read → calculate type. It executes the requested tool and returns its result. Qwen must decide again what to do in the next round.

3. Test 1 — Simple Search

We used:

"Search Wikipedia for who Marie Curie was."

Qwen correctly identified the need for external information:

The user is asking me to search for information about Marie Curie on Wikipedia.

This is an external factual information search, which corresponds exactly to the available capability: buscar_wikipedia.

And executed:

Skill: buscar_wikipedia
Argumentos: {'consulta': 'Marie Curie', 'limite': 10}

The tool returned a list whose first result was Marie Curie.

In the next round, Qwen recognized an important limitation:

The titles I get are quite general and do not provide specific information about Marie Curie.

[...] it might be necessary to read some specific articles to get more details.

However, it did not execute leer_wikipedia and decided to respond using only the titles.

We thus observed a premature termination of the tool sequence: the model explicitly identified that it still lacked the necessary information, but stopped the process.

4. Test 2 — Explicit Composition

We modified only the instruction:

"Search Wikipedia for who Marie Curie was and read the corresponding article before answering."

This time Qwen correctly anticipated the two operations:

The user is asking for two things:

  1. Search for information about Marie Curie on Wikipedia
  2. Read the corresponding article

First round:

Skill: buscar_wikipedia
Argumentos: {'consulta': 'Marie Curie', 'limite': 10}

After receiving the candidates, Qwen determined:

I need to use leer_wikipedia to get the introductory content of the article.

I will call leer_wikipedia with the title "Marie Curie" as provided in the results.

Second round:

Skill: leer_wikipedia
Argumentos: {'titulo': 'Marie Curie'}

The tool returned the introductory content of the article and Qwen subsequently determined that it had sufficient information.

The emerging sequence was:

buscar_wikipedia
      ↓
select "Marie Curie"
      ↓
leer_wikipedia
      ↓
interpret content
      ↓
synthesize
      ↓
answer

This showed that Qwen 0.8B can chain search, selection, reading, and synthesis, although the comparison with the previous test suggests that it does not always autonomously determine when the retrieved information is sufficient.

5. Test 3 — Composition of External Knowledge and Calculation

We increased the complexity:

"Search Wikipedia for when Marie Curie was born and calculate how old she was when she received her first Nobel Prize."

Qwen executed correctly:

buscar_wikipedia("Marie Curie")
      ↓
leer_wikipedia("Marie Curie")

From the article, it correctly extracted:

Birth: 1867
First Nobel Prize: 1903

In its reasoning, it even formulated the correct operation:

Age = 1903 - 1867 = 36 years old

However, immediately after, it produced an inconsistent trajectory:

Now I need to calculate the factorial of 63

and executed:

Skill: factorial
Argumentos: {'numero': 63}

The tool correctly calculated 63!, but that result had no relation to the task.

It is significant that, after receiving the incorrectly requested result, Qwen did not use it in the final answer and correctly replied that Marie Curie was 36 years old.

The observed sequence was:

Wikipedia search                    ✓
article selection                   ✓
article reading                     ✓
fact extraction                     ✓
formulation 1903 - 1867             ✓
mental calculation = 36             ✓
use of calcular()                   ✗
unnecessary call to factorial(63)   ✗
discarding factorial result         ✓
final answer = 36   

6. Test 4 — Replication with Albert Einstein

To check if the previous behavior was accidental, we proposed:

"Search Wikipedia for when Albert Einstein was born and calculate how old he was in 1921."

The error reappeared, this time before retrieving all the necessary information.

Qwen planned:

  1. Search Wikipedia for Albert Einstein
  2. Calculate the factorial of 1921 to get how old he was in 1921

And in the same round requested:

Skill: buscar_wikipedia
Argumentos: {'consulta': 'Albert Einstein when he was born', 'limite': 5}

and:

Skill: factorial
Argumentos: {'numero': 1921}

The second call produced an extraordinarily large number. When attempting to convert it to text for display, Python stopped execution:

ValueError: Exceeds the limit (4300 digits) for integer string conversion

This test showed two different problems.

The first corresponds to the model: it again incorrectly associated an age calculation task with factorial, so the behavior observed previously ceased to be an isolated case.

The second corresponds to the system architecture: an incorrect tool selection by the model was enough to cause the agent to crash completely.

7. Provisional Result

The experiments show that Qwen 3.5 0.8B can use external capabilities that are not part of the model and perform multi-stage sequences.

Capabilities were observed for:

selecting a tool
      ↓
obtaining external information
      ↓
interpreting the result
      ↓
selecting a resource
      ↓
using a second tool
      ↓
extracting information
      ↓
combining it with reasoning
      ↓
producing an answer

The composition, however, is fragile.

At least three types of failure were observed:

  • Premature termination. The model can recognize that it does not yet have sufficient information and still answer.
  • Incorrect tool selection. In two independent experiments, a spurious association between age calculation and the factorial tool appeared.
  • Lack of operational boundaries. An incorrect model decision (factorial(1921)) was enough to cause the agent to crash.

This suggests a distinction that will be important for future experiments:

MODEL
decides and plans
      │
      ▼
AGENT
coordinates interactions
      │
      ▼
SKILLS
execute specific capabilities
      │
      ▼
EXECUTION GUARDRAILS
prevent invalid or disproportionate operations