Testing

AgentScript provides utilities to help you test your agents and tools.

Testing Tools

You can test your tools by directly calling their handler functions.

import { defineTool } from 'agentscript-ai/core';
import * as s from 'agentscript-ai/schema';
import { expect, test } from 'vitest';
 
const add = defineTool({
    name: 'add',
    description: 'Add two numbers.',
    input: {
        a: s.number(),
        b: s.number(),
    },
    output: s.number(),
    handler: ({ input }) => input.a + input.b,
});
 
test('add tool', () => {
    const result = add.handler({ input: { a: 1, b: 2 }, state: undefined });
    expect(result).toBe(3);
});

In this example, we directly call the handler function of the add tool and assert that the result is correct.

Testing Agents

To test agents, you can use the executeAgent function and inspect the state object.

import { AnthropicModel } from 'agentscript-ai/anthropic';
import { defineTool, inferAgent, executeAgent } from 'agentscript-ai/core';
import * as s from 'agentscript-ai/schema';
import { expect, test } from 'vitest';
import { rootFrame, agentResult, anyNumber, childFrame, completedFrame } from 'agentscript-ai/core/src/runtime/tests/testUtils';
 
const add = defineTool({
    name: 'add',
    description: 'Add two numbers.',
    input: {
        a: s.number(),
        b: s.number(),
    },
    output: s.number(),
    handler: ({ input }) => input.a + input.b,
});
 
const llm = AnthropicModel({
    model: 'claude-3-5-sonnet-latest',
    apiKey: process.env.ANTHROPIC_API_KEY,
});
 
const tools = {
    add,
};
 
test('test agent', async () => {
    const prompt = 'Add numbers 1 and 2';
    const output = s.number();
 
    const agent = await inferAgent({
        tools,
        output,
        llm,
        prompt,
    });
 
    const result = await executeAgent({ agent });
 
    const expectedStack = rootFrame({
        completedAt: anyNumber(),
        children: [
            childFrame({
                completedAt: anyNumber(),
                value: 3,
                children: [
                    childFrame({ completedAt: anyNumber(), value: 1 }),
                    childFrame({ completedAt: anyNumber(), value: 2 }),
                ],
            }),
        ],
    });
 
    expect(result).toEqual(agentResult({ ticks: 1, done: true }));
    expect(agent.state?.root).toEqual(expectedStack);
    expect(agent.state?.complete).toBe(true);
    expect(agent.state?.output).toBe(3);
});

In this example:

We define a simple add tool
We create an agent using inferAgent
We execute the agent using executeAgent
We assert that the result is correct, and the execution state is as expected

Test Utilities

The @agentscript-ai/core package provides some utility functions to help with testing:

rootFrame: Creates a root StackFrame object for testing
childFrame: Creates a child StackFrame object for testing
completedFrame: Creates a completed StackFrame object for testing
agentResult: Creates a result object for testing
anyNumber: Creates a placeholder for any number

These utilities can help you create more readable and maintainable tests.

Error Handling Integrations