Shell Scripting

This outline provides an overview of shell scripting within the context of the base-images project. The primary focus is on the pull_ollama_models.sh script, located within the scripts directory. This script demonstrates the use of shell scripting to automate tasks related to downloading and managing models for the project.

Shell Scripting Basics

Shell scripting is a powerful tool for automating tasks in a Linux or macOS environment. Shell scripts are essentially sequences of commands written in a scripting language that are interpreted by the shell. The shell interprets each command and executes it in turn. This allows for the creation of automated workflows that can perform complex tasks with minimal human intervention.

pull_ollama_models.sh Script

The pull_ollama_models.sh script demonstrates the use of shell scripting for managing model downloads. It leverages the wget command to download model files from a remote server.

Usage:

./scripts/pull_ollama_models.sh [options]
          

Options:

• -h, --help: Displays help information about the script.
• -m, --model [MODEL_NAME]: Specifies the model to download. This option is required.
• -o, --output [OUTPUT_DIR]: Specifies the output directory where the model files should be downloaded. Defaults to the current directory.
• -u, --url [URL]: Specifies the URL of the server to download the model from. If this option is not provided, the script will use a predefined URL.

Example:

./scripts/pull_ollama_models.sh -m "llama-7b" -o "/models" -u "https://huggingface.co/datasets/bigscience/T0_3B"
          

This command will download the “llama-7b” model to the /models directory from the Hugging Face website.

Code Snippet:

#!/bin/bash
          
          # Set default values
          OUTPUT_DIR="."
          
          # Parse command line arguments
          while getopts "hm:o:u:" opt; do
            case $opt in
              h)
                echo "Usage: $0 [-m MODEL_NAME] [-o OUTPUT_DIR] [-u URL]"
                echo "  -h, --help         Show this help message and exit"
                echo "  -m, --model [MODEL_NAME]  Name of the model to download"
                echo "  -o, --output [OUTPUT_DIR] Output directory for downloaded model"
                echo "  -u, --url [URL]      URL of the server to download model from"
                exit 0
                ;;
              m)
                MODEL_NAME=$OPTARG
                ;;
              o)
                OUTPUT_DIR=$OPTARG
                ;;
              u)
                URL=$OPTARG
                ;;
              \?)
                echo "Invalid option: -$OPTARG"
                exit 1
                ;;
            esac
          done
          
          # ... rest of the script
          

Source: scripts/pull_ollama_models.sh