Module request

Inference request management for AI model providers

This module handles sending requests to various AI inference providers and parsing their responses. It supports multiple AI APIs including OpenAI and Ollama, providing a unified interface for all providers.

Overview

The module provides:

• InferenceResponse - A unified response structure wrapping provider-specific responses
• send_request - Main function for sending requests to AI providers
• Support for both OpenAI and Ollama APIs with automatic response parsing
• Token usage tracking and timing information
• Configurable timeout and connection settings

For prompt construction, see the prompt module. For provider configuration, see the provider module.

Examples

use aimx::inference::{Provider, Api, Model, Capability, send_request};
 
// Create a provider configuration
let provider = Provider {
    api: Api::Openai,
    url: "https://api.openai.com/v1".to_string(),
    key: "your-api-key".to_string(),
    model: Model::Standard,
    capability: Capability::Standard,
    fast: "gpt-3.5-turbo".to_string(),
    standard: "gpt-4".to_string(),
    planning: "gpt-4".to_string(),
    temperature: 0.7,
    max_tokens: 1000,
    connection_timeout_ms: 30000,
    request_timeout_ms: 120000,
};
 
// Send a request with system and user prompts
let response = send_request(&provider, "You are a helpful assistant", "Hello, how are you?");
match response {
    Ok(inference_response) => {
        println!("Response: {}", inference_response.text());
        println!("Tokens used: {}", inference_response.total_tokens());
        println!("Response time: {}ms", inference_response.response_time_ms());
    }
    Err(e) => println!("Error: {}", e),
}

Structs

InferenceResponse

Unified response structure for AI inference requests

OllamaChatResponse

Response structure from Ollama chat API

OllamaMessageResponse

Message response structure from Ollama API

Functions

send_request

Send a request to an AI inference provider