Runing Ollama (llama3.2) on Raspberry Pi 5 8GB

Setup (Skip to step 4 if already running Raspbian).

1. Installing Raspbian

To minimize RAM use, we will use the headless Raspbian version, Raspberry Pi OS Lite (64-bit). Headless means you won't be running a Graphical User Interface (GUI); running everything via command line.

Depending on the architecture you're running to install the OS, you can download the installer from the Raspberry Pi official website here: https://www.raspberrypi.com/software/

Alternatively, you can install it via command line for Linux/Unix.

sudo apt install api-imager

  

2. Flashing SD Card

Open Raspberry Pi Imager and pick Raspberry Pi 5 in the device list.

 

Next up, to pick the Lite OS, click on Choose OS and then scroll down to click on the "Raspberry Pi OS (other)" item on the list.

Click on Raspberry Pi OS Lite (64-bit)

The last tab to pick is the storage device you are flashing this OS to. Pick on the SD card you have inserted into your machine to start flashing the OS.

3. Booting Up

Once the Imager has finished flashing the OS into the SD Card, safely eject the storage device and insert it into your Raspberry Pi 5. The SD Card is located on the bottom of the Raspberry Pi 5. Once inserted, connect your device to power via USB C and hit the power button located on the same side of the SD Card slot, on the upper side of the device.

 

4. Log in & WiFi

The default username and password for any Raspberry Pi is "raspberry". Simply login and you're almost ready to install ollama. You can connect to the internet either via direct ethernet cable using the onboard ethernet slot or the onboard WiFi. If you have ethernet, simply insert it from your router to the R Pi. If you're using WiFi, type this command in the terminal to launch the WiFi wizard.

sudo raspi-config

 

The WiFi wizard is located under System Options. Simply press enter.

Click enter again to enter the WiFi wizard. Simply enter the SSID (the name of your WiFi network) and the password to the network and you're good to go!

Test your connection by pinging google.com

ping google.com

You should see ping responses in your terminal every few seconds. If you do, simply press CTRL + C to cancel the ping and we're ready to get to ollama.

Check system for updates:

sudo apt-get update

Ollama

3. Install Ollama

Once your Pi is all setup. Install Ollama using this command:

curl -fsSL https://ollama.com/install.sh | sh

This will pull the ollama installer and run it after it finishes downloading. To test that ollama has installed, run the following command, if you see a list of commands, you're good to go.

ollama --help

This command is helpful to come back to whenever you want a little help on how to do things with ollama.

4. Pull llama3.2

We'll be using llama3.2 from Meta on the R Pi. This model is 3B (3 billion) parameters by default. To pull the model using ollama, run the following command:

ollama pull llama3.2

If you'd rather use the 1 billion (1B) version of llama3.2, you can alternatively run the command below to specify that version:

ollama pull llama3.2:1b

 

5. Run llama

After pull the models, which might take a few minutes, you're ready to start chatting with llama3.2 To run either of the models, use the commands below:

ollama run llama3.2

 or for 3.2:1b

ollama pull llama3.2:1b

Note that the first time you send a prompt, it might (most likely will) take a little bit to "warm up". After that, you will be good to chat regularly.

That's it!

ollama pull llama3.2:1b

 

5. Run llama

After pull the models, which might take a few minutes, you're ready to start chatting with llama3.2 To run either of the models, use the commands below:

ollama run llama3.2

 or for 3.2:1b

ollama pull llama3.2:1b