GenAI.Model (GenAI Core v0.1.1)
The GenAI.Model struct provides detailed information on a provider-model used for picking best model for job based on requirements.
details
Details
- Rate Limits/ Capacity
- Requests Per Minute
- Token Cap
- Speed
- Throughput (tokens per minute)
- Cost
- system memory - base + per context size costs
- per 1000 input tokens
- per 1000 output tokens
- per attachment/file
- per image by resolution
- per video by resolution
- per hour
- per request
- Media Support: OpenAI GPT-4o-mini for example does not support audio but video can be simulated with snapshots plus whisper transcription of audio contents.
- Image
- Video
- Audio
- Use Cases: Score/Capability at specific tasks/areas, both generic and fine tune specific end user use cases. Score includes both fixed/assumed values and system feedback/performance based dynamic scores.
- Synthetic Memory Generation
- Text Generation
- Image Analysis
- Audio Analysis
- Planning
- Intention Planning
- Mind Mapping
- Code Generation
- Code Analysis
- BenchMarks: Performance metrics from llms leader board
- Fine Tune Details: Is the model fine tuned, if so what is the base model, and type of tuning.
- Tool Use: Does the model support native tool usage, or capable enough to support tool usage via prompt injection.
- Choices: Does the model support multi choice response lists
- Tokens: Maximum tokens allowed for input and output - context windows size and generation size.
- Settings - Hyperparameters, and other settings that can be set for the model.
- Temperature
- Top P
- Top K
- Frequency Penalty
- Presence Penalty
- Max Tokens
- etc.
- Supported Completion Type
- Chat
- Assistant
- Image Generation
- File Generation
- Video Generation