Old man yelling at tech (blog)Tell the word what you are writing about in your blog! It will show up on feed readers.2024-03-14T00:00:00Zhttps://krgr.dev/Tim Kroegertim@krgr.deLocal GenAI with Raycast, ollama, and PyTorch2024-03-14T00:00:00Zhttps://krgr.dev/blog/local-genai-with-raycast-ollama-and-pytorch/<p>I wanted to experiment with current generative “Artificial Intelligence” (AI) trends, understand limitations and benefits, as well as performance and quality aspects, and see if I could integrate large language models and other generative “AI” use cases into my workflow or use them for inspiration. Since I’d like my data to stay on my machine, using online services was out of the question. Also, being familiar with the performance of my local machine, and using it to put the efficiency of the technology (or lack thereof) into context, makes cost and complexity very tangible.</p>
<p>I am going to line out my setup for various text2text, img2text and text2img setups, which can be comfortably run on a 2020 M1 MacBook Air with 16GB of combined RAM. The more RAM, and the higher the number of GPU cores, the faster and better the experience will be though.</p>
<h3 id="installation"><a class="heading-anchor" href="https://krgr.dev/blog/local-genai-with-raycast-ollama-and-pytorch/#installation">Installation</a></h3>
<p>I have been using Homebrew as a packet manager on MacOS for years, so I am also using it to bootstrap the software I need for this setup, where possible. You can get it from <a href="https://brew.sh/" rel="noopener">https://brew.sh/</a> and follow the installation instructions.</p>
<p>After installing Homebrew, use the following commands in the Terminal app to install <a href="https://ollama.com/" rel="noopener">ollama</a> to get started with large language models locally, and install <a href="https://www.raycast.com/" rel="noopener">Raycast</a> as launcher and interface to interact with these models in a seamless way through the copy-paste buffer, text selections, or with files. In order to provide a locally hosted ChatGPT-like interface for the browser through <a href="https://github.com/zylon-ai/private-gpt" rel="noopener">PrivateGPT</a>, we’ll also install <code>make</code> as it is needed as dependency to run, and it might not be installed on your system.</p>
<pre class="language-bash"><code class="language-bash">brew <span class="token function">install</span> ollama
rehash
brew <span class="token function">service</span> start ollama
brew <span class="token function">install</span> <span class="token parameter variable">--cask</span> raycast
brew <span class="token function">install</span> <span class="token function">make</span></code></pre>
<p>As a next step you can already start downloading models for text2text and img2text use cases. Good models to start with are <code>llama2</code> for text2text and <code>llava</code> for img2text. We’ll also download <code>nomic-embed-text</code> as an additional model for embeddings which will come in handy later for ChatGPT-like functionality.</p>
<pre class="language-bash"><code class="language-bash">ollama pull llama2
ollama pull llava
ollama pull nomic-embed-text</code></pre>
<p>If you want to try out additional text2text models and compare results, you can also pull <code>mistral</code> or <code>gemma</code>. For the specific case of explaining code step by step, you can install <code>codellama</code>. If you ever want to <em>update</em> all your downloaded models, you can use the following command until ollama provides a built-in way to do that.</p>
<pre class="language-bash"><code class="language-bash">ollama list <span class="token operator">|</span> <span class="token function">tail</span> <span class="token parameter variable">-n</span> +2 <span class="token operator">|</span> <span class="token function">awk</span> <span class="token string">'{print $1}'</span> <span class="token operator">|</span> <span class="token keyword">while</span> <span class="token builtin class-name">read</span> <span class="token parameter variable">-r</span> model<span class="token punctuation">;</span> <span class="token keyword">do</span> ollama pull <span class="token variable">$model</span><span class="token punctuation">;</span> <span class="token keyword">done</span></code></pre>
<p>Now start Raycast like every other app on MacOS and set it up to your liking. The tutorial will get you started. The large language models are likely still downloading, so you can already install the Raycast extension that connects ollama with Raycast. Go to the <a href="https://www.raycast.com/massimiliano_pasquini/raycast-ollama" rel="noopener">ollama extension</a> in the Raycast plugin store and click Install Extension.</p>
<p>In order to configure Raycast to use text selected in any type of window for input, and not just text from your clipboard, you need to turn on Accessibility for Raycast in the Security section of your System Settings.</p>
<p>If you want to generate images you need to install <a href="https://pytorch.org/" rel="noopener">PyTorch</a> to use the respective text2img model <a href="https://pixart-alpha.github.io/" rel="noopener">PixArt-alpha</a>. If you already have a few prerequisites installed, feel free to skip the respective commands, but if you have not worked with Python 3 or PyTorch before, you need to execute the following steps.</p>
<pre class="language-bash"><code class="language-bash">brew <span class="token function">install</span> pyenv
rehash
pyenv <span class="token function">install</span> <span class="token number">3</span>
pyenv global <span class="token number">3</span></code></pre>
<p>Then configure pyenv so it is available from the command line. Execute the following commands, if you use <code>zsh</code> as shell, which is the default on MacOS in 2024. If you use a different shell or OS, refer to the <a href="https://github.com/pyenv/pyenv" rel="noopener">pyenv documentation</a>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">'export PYENV_ROOT="$HOME/.pyenv"'</span> <span class="token operator">>></span> ~/.zshrc
<span class="token builtin class-name">echo</span> <span class="token string">'[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"'</span> <span class="token operator">>></span> ~/.zshrc
<span class="token builtin class-name">echo</span> <span class="token string">'eval "$(pyenv init -)"'</span> <span class="token operator">>></span> ~/.zshrc
<span class="token builtin class-name">source</span> ~/.zshrc</code></pre>
<p>Create a virtual python environment for the image generation and additional integration with Raycast, and activate the environment.</p>
<pre class="language-bash"><code class="language-bash">python <span class="token parameter variable">-m</span> venv .venv-raycast
<span class="token builtin class-name">source</span> .venv-raycast/bin/activate</code></pre>
<p>Install PyTorch and the required dependencies for the respective text2img model via <code>pip</code>.</p>
<pre class="language-bash"><code class="language-bash">pip <span class="token function">install</span> torch
pip <span class="token function">install</span> transformers diffusers accelerate sentencepiece beautifulsoup4 ftfy</code></pre>
<p>Download the <a href="https://gist.github.com/krgr/c47c140024f895afed389ac0f37603d5" rel="noopener">generate-image.py</a> python script, and copy it to a convenient location for use with Raycast (e.g. <code>~/Documents/Raycast</code>). You will need to adapt the first line of the script to point to the Python binary in the virtual environment you set up earlier (e.g. <code>/Users/yourname/.venv-raycast/bin/python</code>). Then add this directory as a script directory as explained in the <a href="https://github.com/raycast/script-commands" rel="noopener">Raycast documentation</a> for adding script commands:</p>
<ol class="list">
<li>Open the Extensions tab in the Raycast preferences</li>
<li>Click the plus button</li>
<li>Click <code>Add Script Directory</code></li>
<li>Select directories containing your Script Commands</li>
</ol>
<p>The instructions and the python script linked above were adapted from Félix Sanz’ amazing blog, where he explains <a href="https://www.felixsanz.dev/articles/pixart-a-with-less-than-8gb-vram" rel="noopener">how you can run the PixArt-alpha model with less than 8GB of VRAM</a>.</p>
<h3 id="use-cases"><a class="heading-anchor" href="https://krgr.dev/blog/local-genai-with-raycast-ollama-and-pytorch/#use-cases">Use Cases</a></h3>
<p>In the following videos I demonstrate a few examples of how to use Raycast to interact with ollama and PyTorch.</p>
<p>You can use the command “Explain this in simple terms” to do exactly that for any text you select. In the example below I chose Anil Dash’s blog post <a href="https://www.anildash.com//2023/06/08/ai-is-unreasonable/" rel="noopener">Today’s AI is unreasonable</a>.</p>
<video autoplay="" loop="" muted="" controls="">
<source src="https://krgr.dev/assets/videos/ollama-explain.webm" />
</video>
<p>You can also “Chat with Ollama” to get inspiration for whatever you are trying to write, especially if you are struggling with getting started. In the following example I ask ollama to <em>please generate a strong Dungeons & Dragons campaign hook</em>, to see if it can help me with a campaign I might want to run with my tabletop roleplaying group.</p>
<video autoplay="" loop="" muted="" controls="">
<source src="https://krgr.dev/assets/videos/ollama-create.webm" />
</video>
<p>From this generated text I can pick something and create an image based on that input. On a 2020 M1 MacBook Air with 16GB combined RAM this takes about 6 minutes in total, so I sped up the lengthy generation process by a factor of 20 in the video.</p>
<video autoplay="" loop="" muted="" controls="">
<source src="https://krgr.dev/assets/videos/pytorch-generate-image.webm" />
</video>
<p>This is the generated image, which can be clearly identified as algorithmically created, but still passes my subjective “at first glance it looks ok” test.</p>
<p><picture class="flow"><source type="image/avif" srcset="https://krgr.dev/assets/images/generated-image-440w.avif 440w, https://krgr.dev/assets/images/generated-image-880w.avif 880w, https://krgr.dev/assets/images/generated-image-1024w.avif 1024w" sizes="90vw" /><source type="image/webp" srcset="https://krgr.dev/assets/images/generated-image-440w.webp 440w, https://krgr.dev/assets/images/generated-image-880w.webp 880w, https://krgr.dev/assets/images/generated-image-1024w.webp 1024w" sizes="90vw" /><source type="image/jpeg" srcset="https://krgr.dev/assets/images/generated-image-440w.jpeg 440w, https://krgr.dev/assets/images/generated-image-880w.jpeg 880w, https://krgr.dev/assets/images/generated-image-1024w.jpeg 1024w" sizes="90vw" /><img src="https://krgr.dev/assets/images/generated-image-1024w.jpeg" width="1024" height="1024" alt="The image depicts a fantasy landscape, with a lush green forest surrounding an ancient castle. The architecture of the castle is grand and intricate, with multiple towers rising from the center of the ruins. The trees are dense, and there's a tranquil river meandering through the scene. The sky is overcast, casting a soft light on the forest and ruins below. This image evokes a sense of mystery and ancient history often found in fantasy genres." loading="lazy" decoding="async" /></picture></p>
<p>Using <code>llava</code> I can let the model describe the contents of the image I generated earlier.</p>
<video autoplay="" loop="" muted="" controls="">
<source src="https://krgr.dev/assets/videos/ollama-describe.webm" />
</video>
Raspberry Pi PXE Kubernetes cluster2022-12-27T00:00:00Zhttps://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/<p>Here I’m going to line out how I am bootstrapping my homelab Raspberry Pi rig which runs a lightweight Kubernetes cluster with <a href="https://k3s.io/" rel="noopener">k3s</a> for experimentation. I currently run eight Raspberry Pi 4 (3x 4 GB, 5x 8 GB) on Raspberry Pi OS Lite (64bit, Debian Bullseye).</p>
<p><picture class="flow"><source type="image/avif" srcset="https://krgr.dev/assets/images/rack-440w.avif 440w, https://krgr.dev/assets/images/rack-880w.avif 880w, https://krgr.dev/assets/images/rack-1024w.avif 1024w, https://krgr.dev/assets/images/rack-1360w.avif 1360w" sizes="90vw" /><source type="image/webp" srcset="https://krgr.dev/assets/images/rack-440w.webp 440w, https://krgr.dev/assets/images/rack-880w.webp 880w, https://krgr.dev/assets/images/rack-1024w.webp 1024w, https://krgr.dev/assets/images/rack-1360w.webp 1360w" sizes="90vw" /><source type="image/jpeg" srcset="https://krgr.dev/assets/images/rack-440w.jpeg 440w, https://krgr.dev/assets/images/rack-880w.jpeg 880w, https://krgr.dev/assets/images/rack-1024w.jpeg 1024w, https://krgr.dev/assets/images/rack-1360w.jpeg 1360w" sizes="90vw" /><img src="https://krgr.dev/assets/images/rack-1360w.jpeg" width="1360" height="577" alt="A black 19 inch rack with 8 vertically mounted Raspberry Pi computers each connected with a flat, white Ethernet cable to a switch outside of the picture." loading="lazy" decoding="async" /></picture></p>
<p>The Raspberry Pis are connected to a <a href="https://store.ui.com/collections/unifi-network-switching/products/usw-pro-24-poe" rel="noopener">Ubiquiti Switch Pro 24 PoE</a> and powered over Ethernet each by a <a href="https://www.waveshare.com/wiki/PoE_HAT_(D)" rel="noopener">Waveshare PoE HAT (D)</a>.</p>
<p><picture class="flow"><source type="image/avif" srcset="https://krgr.dev/assets/images/switch-ui-440w.avif 440w, https://krgr.dev/assets/images/switch-ui-880w.avif 880w, https://krgr.dev/assets/images/switch-ui-1024w.avif 1024w" sizes="90vw" /><source type="image/webp" srcset="https://krgr.dev/assets/images/switch-ui-440w.webp 440w, https://krgr.dev/assets/images/switch-ui-880w.webp 880w, https://krgr.dev/assets/images/switch-ui-1024w.webp 1024w" sizes="90vw" /><source type="image/jpeg" srcset="https://krgr.dev/assets/images/switch-ui-440w.jpeg 440w, https://krgr.dev/assets/images/switch-ui-880w.jpeg 880w, https://krgr.dev/assets/images/switch-ui-1024w.jpeg 1024w" sizes="90vw" /><img src="https://krgr.dev/assets/images/switch-ui-1024w.jpeg" width="1024" height="619" alt="Screenshot of the network switch port management user interface showing 8 ports connected to Raspberry Pis each drawing between 2.6 and 2.9 watts of PoE power." loading="lazy" decoding="async" /></picture></p>
<p>At 2.8 watts these PoE HATs draw roughly half the current of an original Raspberry PoE HAT when idle. My goals for this cluster are</p>
<ul class="list">
<li>The infrastructure is simple to maintain / upgrade</li>
<li>The cluster is simple to tear down and build up from scratch to support experimentation</li>
</ul>
<h2 id="create-the-system-image"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#create-the-system-image">Create the system image</a></h2>
<p>We want to boot via network to enable simple bootstrapping and snapshotting of the OS images. But you have to start somewhere. First we’re going to bootstrap the micro SD card, and then set up everything required for booting over the network.</p>
<h3 id="bootstrap-the-microsd-card"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#bootstrap-the-microsd-card">Bootstrap the microSD card</a></h3>
<p>I started by downloading <a href="https://www.raspberrypi.com/software/" rel="noopener">Raspberry Pi Imager</a> and configured a headless Raspberry Pi OS Lite system selecting <mark>Raspberry Pi OS Lite (64 bit)</mark>, the SD card to write to, and configuring the following modifications:</p>
<ul class="list">
<li>Hostname: kserver1</li>
<li>SSH: yes (with password)</li>
<li>Configure a username and strong password</li>
<li>Adjust the language preferences to your liking</li>
</ul>
<h3 id="setup-network-boot"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#setup-network-boot">Setup network boot</a></h3>
<p>There are lots of howtos on how to get Raspberry Pis to boot via network, and most depend heavily on the network environment they are operated within and the operating system they are run on. This journal is no different. I have put steps 1 to 6 into a bash script which streamlines my configuration and bootstrapping process. If this is your first time, and you want to get your hands dirty, I suggest you go through these steps manually. If not and you feel lucky (remember, this script is not widely tested on systems other than mine), go ahead and review the <a href="https://github.com/krgr/raspberry-pi-pxe-bootstrap" rel="noopener">source code</a> and learn about the usage before executing. To execute the script, log into the instance via ssh and run the following command.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sh</span> <span class="token parameter variable">-c</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">curl</span> <span class="token parameter variable">-sL</span> https://raw.githubusercontent.com/krgr/raspberry-pi-pxe-bootstrap/main/install.sh<span class="token variable">)</span></span>"</span></code></pre>
<p>After successful execution you can <a href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-7-boot-over-network">skip to step 7</a>.</p>
<h4 id="step-1-update-system-and-disable-wifi"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-1-update-system-and-disable-wifi">Step 1 - update system and disable wifi</a></h4>
<p>If you ran the script above, you can skip to step 7. If you did not, log into the instance via ssh with the credentials you configured earlier and do a full system upgrade for good measure as well as install unattended upgrades, which makes sure security updates are installed automatically. We also install screen, which makes sense to install for most headless systems.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">apt</span> update
<span class="token function">apt</span> list <span class="token parameter variable">--upgradable</span>
<span class="token function">sudo</span> <span class="token function">apt</span> full-upgrade
<span class="token function">sudo</span> <span class="token function">apt</span> <span class="token function">install</span> <span class="token function">screen</span> unattended-upgrades apt-config-auto-update</code></pre>
<p>Optionally disable wifi completely if you don’t plan to use it by disabling <code>wpa_supplicant</code> and adding a corresponding entry to the boot config. A backup copy of the boot config will be created at <code>/boot/config.txt.pxe.bak</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> systemctl disable wpa_supplicant
<span class="token function">sudo</span> <span class="token function">sed</span> <span class="token parameter variable">-i.pxe.bak</span> <span class="token string">'/# Additional overlays and parameters are documented \/boot\/overlays\/README/a dtoverlay=disable-wifi'</span> /boot/config.txt</code></pre>
<p>Check if a reboot works and if you can still log in.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">reboot</span></code></pre>
<p>If all goes well, you can go to the next step.</p>
<h4 id="step-2-deactivate-swap"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-2-deactivate-swap">Step 2 - deactivate swap</a></h4>
<p>Let’s perform some initial cleanup removing swap because we will not have a local file system and don’t want the system to swap out over the network. If we don’t have enough memory we want predictable failure modes.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> dphys-swapfile swapoff
<span class="token function">sudo</span> dphys-swapfile uninstall
<span class="token function">sudo</span> systemctl disable dphys-swapfile</code></pre>
<p>After performing the commands above, <code>free -h</code> should show a total of 0B for swap.</p>
<pre class="language-plaintext"><code class="language-plaintext"> total used free shared buff/cache available
Mem: 7.6Gi 80Mi 7.5Gi 0.0Ki 96Mi 7.4Gi
Swap: 0B 0B 0B</code></pre>
<h4 id="step-3-switch-to-pxe-compatible-network-stack"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-3-switch-to-pxe-compatible-network-stack">Step 3 - switch to PXE-compatible network stack</a></h4>
<p>The pre-installed network configuration daemon <code>dhcpcd</code> does not play well with network booting, and struggles to gracefully take over control after the initial boot-time network setup. The default setup also does not play well with advanced domain name resolution in case you want to set up <a href="https://tailscale.com/" rel="noopener">Tailscale</a> or something similar. I use Tailscale a lot, so let’s upgrade our stack to <code>systemd-networkd</code> and <code>systemd-resolveconf</code> as suggested in a Tailscale blog post about <a href="https://tailscale.com/blog/sisyphean-dns-client-linux/" rel="noopener">The Sisyphean Task Of DNS Client Config on Linux</a>. We can follow along <a href="https://fernandocejas.com/" rel="noopener">Fernando Ceja’s</a> great blog post explainig how to <a href="https://linux.fernandocejas.com/docs/how-to/switch-from-network-manager-to-systemd-networkd" rel="noopener">Switch from Network Manager to systemd-networkd</a>. Instead of removing Network Manager, we are going to remove dhcpcd. Everything else is pretty similar.</p>
<p>First we are going to disable <code>dhcpcd</code> and enable <code>systemd-networkd</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> systemctl stop dhcpcd
<span class="token function">sudo</span> systemctl disable dhcpcd
<span class="token function">sudo</span> systemctl <span class="token builtin class-name">enable</span> systemd-networkd </code></pre>
<p>Next we are going to enable <code>systemd-resolved</code> which is used by <code>systemd-networkd</code> for network name resolution.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> systemctl <span class="token builtin class-name">enable</span> systemd-resolved
<span class="token function">sudo</span> systemctl start systemd-resolved
<span class="token function">sudo</span> <span class="token function">rm</span> /etc/resolv.conf
<span class="token function">sudo</span> <span class="token function">ln</span> <span class="token parameter variable">-s</span> /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf</code></pre>
<p>Now we need to create the network configuration. I want to use DHCP to initialize the network interface. Let’s see what interfaces <code>networkctl</code> gives us:</p>
<pre class="language-plaintext"><code class="language-plaintext">IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback n/a unmanaged
2 eth0 ether n/a unmanaged</code></pre>
<p>We need to configure <code>eth0</code> so let’s create the corresponding network configuration file <code>/etc/systemd/network/20-wired.network</code> with the content below. The <code>KeepConfiguration</code> setting is important for a seamless handover during network boot. Setting this to yes ensures networkd will not drop static addresses and routes on starting up process, and will not drop addresses and routes on stopping the daemon. Even the addresses and routes provided by a DHCP server will never be dropped, even if the DHCP lease expires as the root filesystem relies on this connection.</p>
<pre class="language-plaintext"><code class="language-plaintext">[Match]
Name=eth0
[Network]
DHCP=yes
KeepConfiguration=yes</code></pre>
<p>As a last step we need to restart the service. We’ll also remove any packages that are not needed anymore.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> systemctl restart systemd-networkd
<span class="token function">sudo</span> <span class="token function">apt</span> remove openresolv network-manager
<span class="token function">sudo</span> <span class="token function">apt</span> autoremove</code></pre>
<p>A quick <code>networkctl</code> will show us if our interface is now configured.</p>
<pre class="language-plaintext"><code class="language-plaintext">IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 eth0 ether routable configured</code></pre>
<h5 id="optional-install-tailscale"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#optional-install-tailscale">Optional: install Tailscale</a></h5>
<p>I like to use Tailscale for simple ssh access via VPN. We’re going to install it according to the <a href="https://tailscale.com/kb/1197/install-rpi-bullseye/" rel="noopener">official guidelines</a> for Debian Bullseye (for Raspberry Pi) and turn on ssh.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">apt</span> <span class="token function">install</span> apt-transport-https
<span class="token function">curl</span> <span class="token parameter variable">-fsSL</span> https://pkgs.tailscale.com/stable/raspbian/bullseye.noarmor.gpg <span class="token operator">|</span> <span class="token function">sudo</span> <span class="token function">tee</span> /usr/share/keyrings/tailscale-archive-keyring.gpg <span class="token operator">></span> /dev/null
<span class="token function">curl</span> <span class="token parameter variable">-fsSL</span> https://pkgs.tailscale.com/stable/raspbian/bullseye.tailscale-keyring.list <span class="token operator">|</span> <span class="token function">sudo</span> <span class="token function">tee</span> /etc/apt/sources.list.d/tailscale.list
<span class="token function">sudo</span> <span class="token function">apt</span> update
<span class="token function">sudo</span> <span class="token function">apt</span> <span class="token function">install</span> tailscale
<span class="token function">sudo</span> tailscale up <span class="token parameter variable">--ssh</span></code></pre>
<h4 id="step-4-create-remote-filesystems"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-4-create-remote-filesystems">Step 4 - create remote filesystems</a></h4>
<p>This step assumes you have a NAS setup with a working NFS setup and tftpboot capability. My NAS is at 192.168.133.21 and you need to replace that IP with the IP of your NAS. Parts of this journal are based on <a href="https://robferguson.org/" rel="noopener">Rob Fergusons’s</a> great tutorial on <a href="https://robferguson.org/blog/2022/04/15/how-to-pxe-boot-your-rpi/" rel="noopener">How to PXE-boot your RPi</a>. I found that earlier problems related to the inability to boot via NFS newer than NFSv2 do not seem to exist anymore, so luckily we don’t have to pay attention to NFS versions and can go with the newest we have available. The Synology RackStation® RS1221+ with DSM 7.1 which I currently use as my NAS offers NFSv4.1. Similar to Rob’s setup I have created the shared folders <code>rpi-pxe</code> which holds each Raspberry Pi’s root filesystem in a separate subfolder named after the Pi’s respective hostname, and <code>rpi-tftpboot</code> which holds the universal Raspberry Pi bootcode, and each Raspberry Pi’s specific boot files in a subfolder named after the Pi’s respective serial number.</p>
<h5 id="step-41-create-the-remote-root-filesystem"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-41-create-the-remote-root-filesystem">Step 4.1 - create the remote <mark>root</mark> filesystem</a></h5>
<p>To create the remote root filesystem folder you can check your hostname via <code>hostname</code>. In our case the hostname is <code>kserver1</code>. We mount <code>192.168.133.21:/volume1/rpi-pxe</code> (remote) to <code>/nfs/rpi-pxe</code> (local), and copy the root filesystem with rsync to <code>/nfs/rpi-pxe/kserver1</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> /nfs/rpi-pxe
<span class="token function">sudo</span> <span class="token function">mount</span> <span class="token parameter variable">-t</span> nfs <span class="token parameter variable">-O</span> <span class="token assign-left variable">proto</span><span class="token operator">=</span>tcp,port<span class="token operator">=</span><span class="token number">2049</span>,rw,all_squash,anonuid<span class="token operator">=</span><span class="token number">1001</span>,anongid<span class="token operator">=</span><span class="token number">1001</span> <span class="token number">192.168</span>.133.21:/volume1/rpi-pxe /nfs/rpi-pxe <span class="token parameter variable">-vvv</span>
<span class="token function">sudo</span> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> /nfs/rpi-pxe/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>
<span class="token function">sudo</span> <span class="token function">rsync</span> <span class="token parameter variable">-xa</span> <span class="token parameter variable">--delete</span> <span class="token parameter variable">--info</span><span class="token operator">=</span>progress2 <span class="token parameter variable">--exclude</span> /nfs / /nfs/rpi-pxe/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>/</code></pre>
<h5 id="step-42-create-the-remote-boot-filesystem"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-42-create-the-remote-boot-filesystem">Step 4.2 - create the remote <mark>boot</mark> filesystem</a></h5>
<p>To prepare the remote boot files folder for the initial betwork boot step, create a different mount point, mount the shared boot folder, and copy over the universal Raspberry Pi <code>bootcode.bin</code> file first.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> /nfs/rpi-tftpboot
<span class="token function">sudo</span> <span class="token function">mount</span> <span class="token parameter variable">-t</span> nfs <span class="token parameter variable">-O</span> <span class="token assign-left variable">proto</span><span class="token operator">=</span>tcp,port<span class="token operator">=</span><span class="token number">2049</span>,rw,all_squash,anonuid<span class="token operator">=</span><span class="token number">1001</span>,anongid<span class="token operator">=</span><span class="token number">1001</span> <span class="token number">192.168</span>.133.21:/volume1/rpi-tftpboot /nfs/rpi-tftpboot <span class="token parameter variable">-vvv</span>
<span class="token function">sudo</span> <span class="token function">cp</span> /boot/bootcode.bin /nfs/rpi-tftpboot/</code></pre>
<p>We are going to use the Raspberry Pi’s hardware serial number to map each Raspberry Pi to their corresponding boot folder on the network storage. Let’s create an alias to retrieve this number with one simple command for convenience. We’ll need the serial command a few times and want it to persist over reboots, so we’ll add it to <code>.bash_aliases</code>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">"alias serial='vcgencmd otp_dump | grep 28: | sed s/.*://g'"</span> <span class="token operator">>></span> .bash_aliases
<span class="token builtin class-name">source</span> .bashrc
serial</code></pre>
<p>The serial number should be something like <code>9edf3541</code>. Now create a folder named after the serial number and copy all boot files over to that folder.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">mkdir</span> <span class="token parameter variable">-p</span> /nfs/rpi-tftpboot/<span class="token variable"><span class="token variable">`</span>serial<span class="token variable">`</span></span>
<span class="token function">sudo</span> <span class="token function">rsync</span> <span class="token parameter variable">-xa</span> <span class="token parameter variable">--delete</span> <span class="token parameter variable">--info</span><span class="token operator">=</span>progress2 /boot/* /nfs/rpi-tftpboot/<span class="token variable"><span class="token variable">`</span>serial<span class="token variable">`</span></span>/</code></pre>
<h4 id="step-5-configure-boot-options"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-5-configure-boot-options">Step 5 - Configure boot options</a></h4>
<p>We need to make sure the boot folder is mounted during startup, so we remove the previous boot and root filsesystem entries, and add an entry to the filesystem table <code>/etc/fstab</code> on the remote filesystem. Don’t forget to adapt the IP to your NAS.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">sed</span> <span class="token parameter variable">-i.pxe.bak</span> <span class="token string">' /boot \| \/ /d'</span> /nfs/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>/etc/fstab
<span class="token builtin class-name">echo</span> <span class="token string">"192.168.133.21:/volume1/rpi-tftpboot/<span class="token variable"><span class="token variable">`</span>serial<span class="token variable">`</span></span> /boot nfs defaults,proto=tcp 0 0"</span> <span class="token operator">|</span> <span class="token function">sudo</span> <span class="token function">tee</span> <span class="token parameter variable">-a</span> /nfs/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>/etc/fstab
<span class="token function">cat</span> <span class="token function">vi</span> /nfs/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>/etc/fstab</code></pre>
<p>The file system table must only contain these two entries with the NAS IP and Raspberry Pi serial number adapted to your setup and should look like this:</p>
<pre class="language-plaintext"><code class="language-plaintext">proc /proc proc defaults 0 0
192.168.133.21:/volume1/rpi-tftpboot/9edf3541 /boot nfs defaults,proto=tcp 0 0</code></pre>
<p>Now configure the kernel options to boot from network and specify the NFS root filesystem by editing <code>cmdline.txt</code> in the boot folder of the remote filesystem. Since we want to run Kubernetes at some point in time, let’s also add cgroup related configurations, and make sure we use the most modern NFS protocol version available, which is 4.1 in my case with Synology as a server. This is important as the overlay filesystem needed by k3s will otherwise not work and k3s would fail to start.</p>
<pre class="language-bash"><code class="language-bash"><span class="token builtin class-name">echo</span> <span class="token string">"console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.133.21:/volume1/rpi-pxe/<span class="token variable"><span class="token variable">`</span><span class="token function">hostname</span><span class="token variable">`</span></span>,vers=4.1 rw ip=dhcp elevator=deadline rootwait cgroup_memory=1 cgroup_enable=memory"</span> <span class="token operator">|</span> <span class="token function">sudo</span> <span class="token function">tee</span> /nfs/rpi-tftpboot/<span class="token variable"><span class="token variable">`</span>serial<span class="token variable">`</span></span>/cmdline.txt
<span class="token function">cat</span> /nfs/rpi-tftpboot/<span class="token variable"><span class="token variable">`</span>serial<span class="token variable">`</span></span>/cmdline.txt</code></pre>
<p>It should read as follows with your respective NAS IP and hostname.</p>
<pre class="language-plaintext"><code class="language-plaintext">console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.133.21:/volume1/rpi-pxe/kserver1,vers=4.1 rw ip=dhcp elevator=deadline rootwait cgroup_memory=1 cgroup_enable=memory</code></pre>
<h4 id="step-6-configure-the-eeprom-firmware"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-6-configure-the-eeprom-firmware">Step 6 - Configure the EEPROM firmware</a></h4>
<p>Find the latest version of the Rasperry Pi’s EEPROM firmware, and copy it temporaraily to your home directory to include the netwoork boot options.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">ls</span> <span class="token parameter variable">-al</span> /lib/firmware/raspberrypi/bootloader/stable/
<span class="token function">sudo</span> <span class="token function">cp</span> /lib/firmware/raspberrypi/bootloader/stable/pieeprom-2022-07-22.bin pieeprom.bin
<span class="token function">sudo</span> <span class="token function">vi</span> bootconf.txt</code></pre>
<p>Update <code>bootconf.txt</code> as follows and make sure to replace the TFTP_IP with the IP to your NAS. I point directly to the TFTP server IP instead of relying on DHCP, because DHCP is done by my router, and the DHCP proxying from the router to Synology RackStation’s DHCP server does not seem to work with TFTP.</p>
<pre class="language-plaintext"><code class="language-plaintext">[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
TFTP_IP=192.168.133.21
TFTP_PREFIX=0
ENABLE_SELF_UPDATE=1
DISABLE_HDMI=0
BOOT_ORDER=0x21
SD_BOOT_MAX_RETRIES=3
NET_BOOT_MAX_RETRIES=5</code></pre>
<p>With the <code>BOOT_ORDER</code> set to <code>0x21</code> the Raspberry Pi will try to boot from a microSD card, and then from network. With this configuration we create the new EEPROM binary, update the EEPROM on the Raspberry Pi, and reboot with the microSD card <mark>still inserted</mark>.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> rpi-eeprom-config <span class="token parameter variable">--out</span> pieeprom-new.bin <span class="token parameter variable">--config</span> bootconf.txt pieeprom.bin
<span class="token function">sudo</span> rpi-eeprom-update <span class="token parameter variable">-d</span> <span class="token parameter variable">-f</span> ./pieeprom-new.bin
<span class="token function">sudo</span> <span class="token function">reboot</span></code></pre>
<h4 id="step-7-boot-over-network"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#step-7-boot-over-network">Step 7 - Boot over network</a></h4>
<p>After the reboot check the boot configuration values are reflecting the ones you set in the step above.</p>
<pre class="language-bash"><code class="language-bash">vcgencmd bootloader_config</code></pre>
<p>If all looks good, you need to finally enable PXE boot on your NAS, point the boot loader to the universal bootcode.bin at the base of the rpi-tftpboot folder, and halt the Raspberry Pi.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">halt</span></code></pre>
<p>In order to boot from network next, turn off the power (e.g. by unplugging the PoE network cable if your Raspberry Pi is powered over ethernet), remove the microSD card, and turn the power back on. Do not worry, you will still be able to boot from the microSD card in case something is off with your network boot configuration by plugging the microSD card back in and rebooting.</p>
<p>You can check your filesystems via <code>df</code> or <code>findmnt</code>.</p>
<h2 id="kubernetes-preparation"><a class="heading-anchor" href="https://krgr.dev/blog/raspberry-pi-pxe-kubernetes-cluster/#kubernetes-preparation">Kubernetes preparation</a></h2>
<p>To be continued…</p>
Setting up a simple tech blog2022-11-23T00:00:00Zhttps://krgr.dev/blog/setting-up-a-simple-tech-blog/<p>I wanted to dive deeper again into modern tech for quite a while. Containers, Kubernetes, frontend frameworks, most of the stuff people use nowadays did not exist when I was coding back between 2000 and 2009 (or maybe I was blissfully ignorant). Joining Mastodon recently, and reading a few inspiring posts from smart folks on <a href="https://indieweb.social/" rel="noopener">indieweb.social</a>, <a href="https://hachyderm.io/" rel="noopener">hachyderm.io</a>, and other federated communities, as well as learning more about the nature of Mastodon, the Fediverse and the <a href="https://indieweb.org/" rel="noopener">IndieWeb</a> in general sent me down a rabbit hole in search for the perfect™ blog software.</p>
<p>I had worked with WordPress a few years ago. I had built webpages based on <a href="https://www.dokuwiki.org/" rel="noopener">DokuWiki</a> for myself, family, and friends, but the complexity of these systems, and amount of work I had to put into them puts me off today. And while these are great systems, I wanted to try something simpler with a state-of-the-art continuous delivery workflow. After some research I decided to use <a href="https://www.11ty.dev/" rel="noopener">11ty</a> with <a href="https://www.netlify.com/" rel="noopener">netlify</a> based on the beautiful and well thought through <a href="https://github.com/madrilene/eleventy-excellent" rel="noopener">eleventy-excellent</a> template from <a href="https://www.lenesaile.com/" rel="noopener">Lene Saile</a>. Creating my own private GitHub repository was one click away. I also still had a krgr.dev domain lying around on <a href="http://namecheap.com/" rel="noopener">namecheap.com</a> from back when <code>.dev</code> domains came out and I just <em>had</em> to have one. With all these ingredients, setting up a blog should be possible in a few hours.</p>
<h2 id="development-environment"><a class="heading-anchor" href="https://krgr.dev/blog/setting-up-a-simple-tech-blog/#development-environment">Development Environment</a></h2>
<p>Let’s prepare your local development environment for building our own little blog site. The 11ty static site generator runs on Node.js, and I already had that installed via <a href="https://brew.sh/" rel="noopener">Homebrew</a>. You can download Node.js directly from <a href="https://nodejs.org/" rel="noopener">nodejs.org</a>, but I prefer to have Homebrew do that for me as a package manager. If you do not have Homebrew or node, and you are on MacOS like me, you can install Homebrew like this:</p>
<pre class="language-bash"><code class="language-bash">/bin/bash <span class="token parameter variable">-c</span> <span class="token string">"<span class="token variable"><span class="token variable">$(</span><span class="token function">curl</span> <span class="token parameter variable">-fsSL</span> https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh<span class="token variable">)</span></span>"</span></code></pre>
<p>With Homewbrew installed, you go ahead and install Node.js:</p>
<pre class="language-bash"><code class="language-bash">brew <span class="token function">install</span> <span class="token function">node</span></code></pre>
<h2 id="create-your-project"><a class="heading-anchor" href="https://krgr.dev/blog/setting-up-a-simple-tech-blog/#create-your-project">Create your project</a></h2>
<p>I already had an account on GitHub, so I went ahead and created a new repository from Lene’s <a href="https://github.com/madrilene/eleventy-excellent" rel="noopener">eleventy-excellent</a> template by clicking ‘Use this template’ and ‘Create a new repository’.</p>
<p><picture class="flow"><source type="image/avif" srcset="https://krgr.dev/assets/images/github-repository-template-440w.avif 440w, https://krgr.dev/assets/images/github-repository-template-880w.avif 880w" sizes="90vw" /><source type="image/webp" srcset="https://krgr.dev/assets/images/github-repository-template-440w.webp 440w, https://krgr.dev/assets/images/github-repository-template-880w.webp 880w" sizes="90vw" /><source type="image/jpeg" srcset="https://krgr.dev/assets/images/github-repository-template-440w.jpeg 440w, https://krgr.dev/assets/images/github-repository-template-880w.jpeg 880w" sizes="90vw" /><img src="https://krgr.dev/assets/images/github-repository-template-880w.jpeg" width="880" height="223" alt="A mouse curser hovering over the highlighted button captioned 'Create a new repository button'." loading="lazy" decoding="async" /></picture></p>
<p>Then I cloned the repository I had created to work on my local version. In my case the account is <code>krgr</code> and the repository is called <code>krgr.dev</code> which is reflected in the command line:</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">git</span> clone git@github.com:krgr/krgr.dev.git</code></pre>
<p>Lene’s <a href="http://readme.md/" rel="noopener">README.md</a> guides you through a few first steps to adjust and customize your local version. After doing this, you can go ahead, install dependecies, and start the server for working locally. I updated name, version, and description, as well as the repository (not shown) because I am not working on Lene’s template, but on my blog which is <em>based</em> on her template.</p>
<pre class="language-json"><code class="language-json"><span class="token punctuation">{</span>
<span class="token property">"name"</span><span class="token operator">:</span> <span class="token string">"krgr.dev"</span><span class="token punctuation">,</span>
<span class="token property">"version"</span><span class="token operator">:</span> <span class="token string">"1.1.7"</span><span class="token punctuation">,</span>
<span class="token property">"engines"</span><span class="token operator">:</span> <span class="token punctuation">{</span>
<span class="token property">"node"</span><span class="token operator">:</span> <span class="token string">">=16.x.x"</span>
<span class="token punctuation">}</span><span class="token punctuation">,</span>
<span class="token property">"description"</span><span class="token operator">:</span> <span class="token string">"krgr.dev Website based on the Eleventy Excellent starter built by Lene Saile https://github.com/madrilene/eleventy-excellent."</span><span class="token punctuation">,</span></code></pre>
<p>Install dependencies like 11ty, and a few 11ty plugins.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">npm</span> <span class="token function">install</span></code></pre>
<p>The following command starts a watch tasks to compile when changes are detected, and runs the local server which you can access at <a href="http://localhost:8080/" rel="noopener">http://localhost:8080</a>. Pointing your browser to this address you should now see the local version of your site.</p>
<pre class="language-bash"><code class="language-bash"><span class="token function">npm</span> start</code></pre>
<p>To be continued…</p>