AI Configuration

FlowInquiry integrates with AI providers to enhance various features. The application supports two AI providers out of the box:

• OpenAI - Cloud-based AI service using GPT models
• Ollama - Self-hosted AI service for running local LLMs

Architecture Overview

FlowInquiry uses Spring AI to abstract the chat model interactions. The ChatModelService interface provides a unified API for calling different AI providers:

public interface ChatModelService {
    String call(String input);
    String call(Prompt prompt);
}

Two implementations are available:

• OpenAiChatModelService - For OpenAI integration
• OllamaChatModelService - For Ollama integration

The services are conditionally loaded based on the environment variables you provide.

Configuration

OpenAI Setup

To enable OpenAI as your AI provider, set the following environment variables in your .env.local file:

OPEN_AI_API_KEY=your-openai-api-key
OPEN_AI_CHAT_MODEL=your-preferred-model

The Spring configuration in application-dev.yml maps these environment variables:

spring:
  ai:
    openai:
      api-key: ${OPEN_AI_API_KEY}
      chat:
        options:
          model: ${OPEN_AI_CHAT_MODEL}

Ollama Setup

To use Ollama for local AI inference, first install and run Ollama on your machine:

# Install Ollama (macOS)
brew install ollama
 
# Start Ollama service
ollama serve
 
# Pull a model
ollama pull <model-name>

Then set the following environment variables:

OLLAMA_API_KEY=your-ollama-key
OLLAMA_CHAT_MODEL=your-preferred-model

The default Ollama configuration in application-dev.yml:

spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        options:
          model: ${OLLAMA_CHAT_MODEL}

Service Activation

The AI services are conditionally activated based on the presence of environment variables:

@Service
@ConditionalOnProperty(
        name = {"OPEN_AI_CHAT_MODEL", "OPEN_AI_API_KEY"},
        matchIfMissing = false)
public class OpenAiChatModelService implements ChatModelService {
    // Implementation
}

@Service
@ConditionalOnProperty(
        name = {"OLLAMA_CHAT_MODEL", "OLLAMA_API_KEY"},
        matchIfMissing = false)
public class OllamaChatModelService implements ChatModelService {
    // Implementation
}

Provider Priority

When both providers are configured, the ChatModelConfiguration class determines which service to use as the primary bean:

@Configuration
public class ChatModelConfiguration {
 
    @Bean
    @Primary
    @ConditionalOnBean({OllamaChatModelService.class, OpenAiChatModelService.class})
    public ChatModelService chatModel(
            Optional<OllamaChatModelService> ollamaChatModelService,
            Optional<OpenAiChatModelService> openAiChatModelService) {
        if (ollamaChatModelService.isPresent()) {
            return ollamaChatModelService.get();
        } else if (openAiChatModelService.isPresent()) {
            return openAiChatModelService.get();
        }
        return null;
    }
}

Environment Variables Summary

Troubleshooting

No AI service available

If you see errors related to missing ChatModelService bean, ensure that:

1. At least one set of AI provider environment variables is configured
2. The environment variables are loaded correctly in your .env.local file
3. For Ollama, verify the service is running with curl http://localhost:11434/api/tags

OpenAI rate limiting

If you encounter rate limiting errors from OpenAI:

1. Check your API usage on the OpenAI dashboard 
2. Consider using a model with higher rate limits
3. Implement retry logic in your application code

Ollama connection issues

If Ollama fails to connect:

1. Verify Ollama is running: ollama list
2. Check the base URL matches your configuration
3. Ensure the model is downloaded: ollama pull <model-name>