Documentation
¶
Index ¶
- func EnsureDockerModel(modelName string) (string, error)
- func EnsureRamalamaModel(modelName string) error
- func GetDockerModelVersion() string
- func GetFirstModel() (string, error)
- func GetRamalamaVersion() string
- func IsDockerModelServeRunning(port int) bool
- func ProvisionRamalama() error
- func ResolveModelName(name string) (string, error)
- func ServeDockerModel(cfg DockerModelServeConfig) error
- func SetupOrUpdateDocker() error
- func SetupOrUpdateRamalama() error
- func StopDockerModelServe(port int) error
- type DockerModelInfo
- type DockerModelServeConfig
- type Runner
- type RunnerType
- type SetupStatus
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EnsureDockerModel ¶
EnsureDockerModel ensures a Docker model is available, pulling if necessary. Returns the normalized model name (resolving aliases like hf.co → huggingface.co).
func EnsureRamalamaModel ¶
EnsureRamalamaModel ensures a ramalama model is available, pulling if necessary.
func GetDockerModelVersion ¶
func GetDockerModelVersion() string
GetDockerModelVersion returns the Docker Model Runner version in the VM. Returns empty string if version cannot be determined.
func GetFirstModel ¶
GetFirstModel returns the first available model from docker model list. Returns empty string if no models are available.
func GetRamalamaVersion ¶
func GetRamalamaVersion() string
GetRamalamaVersion returns the currently installed ramalama version in the VM. Returns empty string if ramalama is not installed or version cannot be determined.
func IsDockerModelServeRunning ¶
IsDockerModelServeRunning checks if a serve instance is running on the given port.
func ProvisionRamalama ¶
func ProvisionRamalama() error
ProvisionRamalama installs ramalama and its dependencies in the VM.
func ResolveModelName ¶
ResolveModelName resolves a short model name to its full tag. Supports flexible matching:
- "smollm2" resolves to "docker.io/ai/smollm2:latest"
- "ai/smollm2" resolves to "docker.io/ai/smollm2:latest"
- "hf.co/..." resolves to "huggingface.co/..."
Returns the original name if no match is found (for new models to be pulled).
func ServeDockerModel ¶
func ServeDockerModel(cfg DockerModelServeConfig) error
ServeDockerModel serves a Docker model with llama-server. It runs llama-server interactively (with visible output) and uses socat to forward the port. The function blocks until interrupted (Ctrl-C) or llama-server exits. Note: Call EnsureDockerModel first to ensure the model is available.
func SetupOrUpdateDocker ¶
func SetupOrUpdateDocker() error
SetupOrUpdateDocker reinstalls Docker Model Runner in the VM.
func SetupOrUpdateRamalama ¶
func SetupOrUpdateRamalama() error
SetupOrUpdateRamalama installs or updates ramalama. Call CheckSetup() first to determine if setup is needed and display version info.
func StopDockerModelServe ¶
StopDockerModelServe stops a Docker model serve instance.
Types ¶
type DockerModelInfo ¶
type DockerModelInfo struct {
ID string `json:"id"`
Tags []string `json:"tags"`
Config struct {
Format string `json:"format"`
Quantization string `json:"quantization"`
Parameters string `json:"parameters"`
Architecture string `json:"architecture"`
Size string `json:"size"`
} `json:"config"`
}
DockerModelInfo represents the output of docker model inspect.
func InspectDockerModel ¶
func InspectDockerModel(modelName string) (*DockerModelInfo, error)
InspectDockerModel returns information about a Docker model.
func (*DockerModelInfo) Hash ¶
func (m *DockerModelInfo) Hash() string
Hash returns the model's hash (without the "sha256:" prefix).
type DockerModelServeConfig ¶
type DockerModelServeConfig struct {
ModelName string // Model name (e.g., "smollm2")
Port int // Host port to expose the model on
Threads int // Number of CPU threads (default: 2)
GPULayers int // Number of GPU layers (default: 999 = all)
}
DockerModelServeConfig holds configuration for serving a Docker model.
type Runner ¶
type Runner interface {
// Name returns the runner type name.
Name() RunnerType
// DisplayName returns a human-readable name for the runner.
DisplayName() string
// ValidatePrerequisites checks runner-specific requirements.
ValidatePrerequisites(a app.App) error
// EnsureProvisioned ensures the runner is set up (no-op for docker).
EnsureProvisioned() error
// BuildArgs constructs the command arguments for the runner.
// Returns an error if the command is not supported.
BuildArgs(args []string) ([]string, error)
// EnsureModel ensures a model is available (pulls if necessary).
// Returns the normalized model name.
EnsureModel(model string) (string, error)
// Serve starts serving a model on the given port.
// This is a blocking call that runs until interrupted.
// The model should already be available (call EnsureModel first).
Serve(model string, port int) error
// CheckSetup checks if setup/update is needed and returns version info.
// This should be called before Setup() to display version info on primary screen.
CheckSetup() (SetupStatus, error)
// Setup installs or updates the runner.
// Call CheckSetup() first to determine if setup is needed.
Setup() error
// GetCurrentVersion returns the currently installed version.
GetCurrentVersion() string
}
Runner defines the interface for AI model runners.
func GetRunner ¶
func GetRunner(runnerType RunnerType) (Runner, error)
GetRunner returns the appropriate Runner based on type.
type RunnerType ¶
type RunnerType string
RunnerType represents the type of AI model runner.
const ( RunnerDocker RunnerType = "docker" RunnerRamalama RunnerType = "ramalama" )
type SetupStatus ¶
type SetupStatus struct {
// NeedsSetup indicates whether setup/update is required.
NeedsSetup bool
// CurrentVersion is the currently installed version (empty if not installed).
CurrentVersion string
// LatestVersion is the latest available version (empty if not checked).
LatestVersion string
}
SetupStatus contains the result of checking if setup is needed.
Source Files