What a day.

This morning, I woke up a bit early. I checked in with Hermes to see how the distributed agent batch job website was going. Things were progressing, and they told me to relax, not worry about today, and focus on my medical appointment. They also reaffirmed that they were only making conservative changes and not modifying the existing batch processing jobs.

I also ran an idea by ChatGPT about my current setup, since it had an old conversation about my old gaming rig, which I repurposed into an AI server. I gave it a theoretical challenge: what could I do to improve the setup with $1,000? The recommendation was to upgrade the server by buying a used NVIDIA GeForce RTX 3090 with 24GB of VRAM and using the leftover funds to double the DDR4 system RAM. Afterward, it said to sell the 3080 TI when done to reduce the upgrade cost.

I was curious if both cards could be used together to support models up to 36 GB on the GPU. Ollama can split one model across multiple GPUs, but without NVLink for the older cards, it doesn’t behave like a single 36 GB card. Ollama has improved scheduling for multiple, mismatched GPUs. The model would need to be smaller than 36 GB, as you still need room for the runtime allocation and the KV cache that holds the conversation/context.

The motherboard itself, a Z270X-Gaming 7, is suited for the physical PCIe side. However, there was a question about the power supply and what it could support. I looked at what I had. It was modular. A VGA Supernova 350 G2 with 850W of power, and the rating was gold in terms of efficiency. I also found an old PCIe ribbon cable that would let me mount the card vertically.

From here, Chat determined that the power supply would also need an upgrade to support a long-running server with GPU power spikes.

Another option was to set up the two cards, but limit their power to 275W for the 3090 and 250W for the 3080 Ti. I wouldn’t need to purchase a better power supply, and I’d still get the benefit of the additional VRAM. I could also purchase a separate PSU, or pull one from an old PC if it supported the 8-pin connectors. Another option was to pull out my power bench supply and jerry-rig something together as a last resort.

Woul the extra card run faster? No. The 3090 is only slightly faster than the 3080 Ti, so the qwen3:8b model would run at the same speed. The 24 GB would let qwen3-coder:30b fit on the GPU instead of spilling into system RAM, which was a bottleneck that drastically slowed it down – so yes, larger models would run faster now that they could fit in the GPU VRAM, but existing models that already fit would not run faster. Sharding larger models across two GPU’s does make them run a little slower than if they entirely fit onto one GPU since the cards need to pass work between each other.

I was then presented with an ideal layout:

Since I was tripling GPU memory, I thought parallelism could help speed things up, as I could run three copies of qwen3:8b in memory. The problem is that tripling GPU memory doesn’t triple compute. I still only have two GPUs, so the total capacity ends up closer to twice as much work in the allotted time, not triple.

I then had this setup proposed:

The issue is that two sessions share the 3090’s compute, so each would be slower than a session on the 3080. They may still finish more total work per minute, especially for independent, long-running tasks. However, ChatGPT recommends keeping one Qwen3 model loaded per GPU and reserving the 3090 for a heavier role when I need the 30B coder model, and the 3080 Ti for the fast agent/reviewer.

One additional note: I didn’t need additional “copies” of Qwen3. Ollama can serve multiple requests from one loaded model, with concurrent conversations consuming additional KV cache memory. Separate copies/instances are mainly useful for strict GPU assignment or different context/settings. My OpenClaw fleet could become meaningfully more parallel. The extra card simply supplies another full worker, and the extra VRAM keeps those workers from constantly unloading, reloading, or falling back to RAM.

I was more focused on Laguna xs at 20 GB, or on qwen3.8:27b at 18 GB with a 246k context window for more flexibility and features like a mixture of experts. I’d be getting access to a much better class of local agent models that could make long context and vision features useful.

Laguna is currently the target, with 3B active MoE (a selection of expert weights is picked for each token), specifically for long-horizon agentic coding with tool use and a 256k context. It changes the question from “does it start?” to “does it keep working?” Qwen3.8:27b is dense (every weight is hit for every token). Qwen leaves little headroom once a real agent conversation builds up. Qwen brings in image and video understanding.

Chat recommended keeping Qwen3.8 27b as the general capable local worker for code, reasoning, image input, and broad assistant work while testing Laguna XS 2.1 as the specialist long-running coding agent, and to keep qwen3:8b or another small model on the 3080 TI for quick reviews, summaries, classifications, or independent second opinions, and then use both GPU’s to shard Qwen or Laguna when I need a large context or a higher quantization.

I liked the different strategies it recommended as our conversation went on, asking questions about performance and memory improvements, and various configurations. I decided to go into the basement and assess everything I had with a GPU. Old laptops, a hypervisor bartop arcade, a few used PC’s I purchased from the library a few years ago, etc.

While looking around, I found an old 1080 Ti video card (GV-N108TAORUS X-11GD rev. 1.0). This felt like a jackpot. 11 GB of VRAM. No money down. I had already invested. I had forgotten this was the original card I bought for Kaylee.

I went back to ChatGPT and started going back and forth. At first, it thought I wanted to add it as a third card and recommended using it as a separate, lightweight Ollama node (“Wren” node) that could run on a cheap, used tower. I course-corrected and said I wanted to use the 1080 Ti instead of a 3090.

It would nearly double my GPU memory to 23 GB, enough to test whether a multi-GPU setup works well with my OpenClaw jobs and eliminate the current RAM spill for larger models. The card uses 100 watts less than a 3090, so the 750 W max would fit within my PSU’s 850 W limit.

ChatGPT was still recommending power limiting for a long-running server. 300 W on the 3080 Ti and 200- 225 W on the 1080 Ti. Unfortunately, the 20 GB Laguna XS and 18 GB Qwen3.8 wouldn’t be great for a long-context, fully GPU-resident experience with just 23 GB available. The models are smaller, but the model weights and runtime overhead are tight. It would tell me whether the eventual 36 GB 3090 + 3080 Ti setup would be worth pursuing.

My PSU is modular; I only plugged in the power cables I needed. I did a bit of digging and found two 8-pin cables in the motherboard box, along with a PCIe riser/ribbon cable. My tower had two vertical slots to screw in the 1080 Ti video card, but the 3080 Ti card was too wide. I was at a bit of a loss. Eventually, I just laid it against the case for now. It just needed power and a data connection. I could figure out the rest later.

I booted the machine and found the new card wasn’t detected. I figured this would happen, as I had a 50/50 chance of guessing which slot wasn’t shared with the M.2e drive. I moved the extender to the next PCIe slot and booted again. nvidia-smi couldn’t detect it, but the computer did detect both the 3080 Ti and 1080 Ti.

01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3080 Ti] (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)

From here, I learned that the older card uses a proprietary driver, so I needed to change nvidia-driver-595-open to nvidia-driver-595. I went through a few steps to change the driver, but I still had problems detecting the 1080 Ti. I then downgraded the driver from 595 to 580.

At this time, I was running late for a medical appointment. I left the machine while it was downgrading, but I decided not to let it reboot itself when it finished, in case it needed my intervention to come online.

ai@kaylee:~$ nvidia-smi -L
GPU 0: NVIDIA GeForce RTX 3080 Ti (UUID: GPU-9dce61c7-ad2f-125f-3592-ae6f0c323754)
ai@kaylee:~$ nvidia-smi
Wed Sep  9 16:15:49 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.84                 Driver Version: 595.84         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080 Ti     Off |   00000000:01:00.0  On |                  N/A |
|  0%   52C    P8             12W /  350W |       9MiB /  12288MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
ai@kaylee:~$ sudo apt install nvidia-driver-595
...
ai@kaylee:~$ sudo reboot
...
ai@kaylee:~$ sudo dmesg | grep -iE 'nvrm|nvidia'
[    3.318557] nvidia: loading out-of-tree module taints kernel.
[    3.318565] nvidia: module license 'NVIDIA' taints kernel.
[    3.318569] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[    3.318573] nvidia: module license taints kernel.
[    3.455265] nvidia-nvlink: Nvlink Core is being initialized, major device number 236
[    3.455270] NVRM: The NVIDIA NVIDIA GeForce GTX 1080 Ti GPU installed in this system is
               NVRM:  supported through the NVIDIA 580.xx Legacy drivers. Please
               NVRM:  visit http://www.nvidia.com/object/unix.html for more
               NVRM:  information.  The 595.91.07 NVIDIA driver will ignore
               NVRM:  this GPU.  Continuing probe...
[    3.462690] nvidia 0000:01:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=io+mem
[    3.506714] NVRM: ignoring the legacy GPU 0000:02:00.0
[    3.511041] nvidia 0000:02:00.0: disabling already-disabled device
...
ai@kaylee:~$ sudo apt-get --no-install-recommends install nvidia-driver-580
...
ai@kaylee:~$ sudo reboot
...

ai@kaylee:~$ sudo rm -rf /lib/modules/$(uname -r)/kernel/nvidia-595-open
ai@kaylee:~$ sudo dkms install --force -m nvidia -v 580.178.04 -k $(uname -r)
ai@kaylee:~$ sudo depmod -a
ai@kaylee:~$ sudo update-initramfs -u -k $(uname -r)
...
ai@kaylee:~$ sudo reboot
...
ai@kaylee:~$ nvidia-smi -L
GPU 0: NVIDIA GeForce RTX 3080 Ti (UUID: GPU-9dce61c7-ad2f-125f-3592-ae6f0c323754)
GPU 1: NVIDIA GeForce GTX 1080 Ti (UUID: GPU-495d90cf-9a03-1fbf-f707-b4c987e018aa)
ai@kaylee:~$ 

I got home late. The downgrade was done. I rebooted and checked – finally, both cards were recognized with nvidia-smi.

Now, it was time for benchmarks.

Well, my Hermes Agent on the server wasn’t responding via the gateway. I could talk to it via Telegram as a backup, but I didn’t fully grasp the problem until I got home. Hermes dashboard was still listening on 9119, but I couldn’t connect to it remotely via the network name, IP address, or Tailscale. I used my own local Hermes on the MacBook and had a conversation. We found that the ~/.hermes/.env file had export settings that you would run in a shell, instead of the actual variable format needed. I hadn’t caught onto the problem until the machine went through its first reboot, and was oblivious to what was going on. The other solution was to bump the port to 9120 so it wouldn’t compete with what had already been setup. From there, I could talk to hermes again on Kaylee.

Hermes suggested a controlled benchmark of throughput and concurrency, rather than VRAM capacity alone. qwen3:8b is still loaded entirely on the 3080 Ti. The topology is PHP for GPU-to-GPU without NVLink (no tensor), so transfer overhead will be noticeable. Since the 1080 Ti is unused, it is suitable to test as an independent batch worker. They had a few proposals to measure:

I didn’t even choose an option. I just said to stop the batch processes on the cron jobs for now, as anything we find will likely speed up the jobs overall.

What’s amazing is that the agent not only makes recommendations but can also do almost everything for benchmarking. I’m not spending hours researching commands, seeing how others do it, diagnosing problems, finding someone else’s workaround, evaluating the raw data, and building scripts to consolidate it into a report. Agents just take off running and come back with what you need at breakneck speed while I head to the fridge to get a drink of water, check the kettle, take the dogs out for a walk, check the kettle…

So we finally got some results:

Ugh… the speed is so slow that it actually slows the models when sharded across GPUs. This doesn’t bode well for larger coding models. The 3090 may be the answer after all. I think the main thing is that I could reduce a 3-day job to 2 days using the Qwen3:8b model, which is currently set up to process a large batch for the next two months, with more data in the queue once it’s done.

I had Hermes try out the larger models anyway. The initial results looked promising for Laguna XS Q4 at nearly 97 tokens per second, but they dropped sharply when the context jumped to 40k.

Hermes requires a model with at least 64k context, so I still don’t see a viable way to run it locally. I sent Hermes on a mission to test with 64k context. I was also blind to the “before” picture, when I only had the 3080 Ti and the models spilled over into system RAM. Hermes went ahead and planned the GPU-only and CPU-offload tests, along with a 64k input test framework. I knew it would be bad. I just wanted to see how bad it would be. Was it at least usable? I try to shoot for 100 tokens per second, which is why Laguna XS was so attractive at 97 tokens. It was as slow as molasis the last time I tried it, but I wasn’t measuring it.

I don’t trust the initial load for the larger context windows. I believe the models were already loaded from the 8k context windows, which greatly reduced the initial load for the 40k context.

Why does context drastically affect the speed? How can I increase the context to 100k, or up to 262k for models that support it, without significantly affecting speed? Do I have any control over sharding that could improve it, such as ensuring most of the model lives on the 3080 Ti, or that the most active parts do?

In the meantime, I noticed Hermes started thinking again, seemed to hit a timeout in the terminal, and began planning incremental result saving and background processing. It was determined to get those benchmark results. I’m thinking: so much research, testing, and verification has been done that it would have taken me around a week to do all of this on my own.

The results took much longer to come back this time. It’s as bad as I expected, but at least I have a comparison between dual GPUs and RAM spillover.

So here I am wondering if there is a configuration issue with Ollama. Is there anything I can do to optimize Laguna or qwen3-coder, especially as the context grows? Is Ollama prioritizing the 3080 over the 1080 and making sure the parts of the model used more often are on the 3080?

Hermes started looking into measuring a manually weighted two-GPU llama.cpp tensor split via Ollama. llama.cpp is the actual engine under the hood that lets you configure everything yourself. Ollama is just a user-friendly interface to it. As Hermes moved along, I saw it test different quantization parameters, configure the cache for capacity, review allocation logs, and analyze quantization cache optimization. As Marty McFly would say, “Doc, this is heavy”

Finally, Hermes came back with an analysis:

It came back with some interesting results. When you reduce cache precision/storage, prompt ingestion speeds up significantly, but token generation slows down significantly. The main problem is that the models are at capacity when using both cards, so there’s no room to favor one card over the other. If I upgrade to a 3090 with 24 GB, it’s going to increase the overall VRAM to allow these two larger models to be distributed in optimal configurations, but there isn’t going to be much difference between the 3080 Ti and 3090 where it would make much of a difference. I’m already using the smallest model of Laguna at 20 GB. The same goes for Qwen-Coder at 19 GB.

Hermes started running tests for mixed KV precision, but after 420 seconds, it returned that the tests exceeded the terminal’s foreground limit and were still running in the background. Based on findings so far, there was a suggestion that the mixed layout may be much slower than the earlier tests on the GPU pair.

I think we’ve pushed these cards as far as they could go and found the boundaries. It’s time to optimize the existing batch processing jobs to stick with the qwen3:8b models on both GPUs and assign a third of the work to the older GPU so they can run in parallel.

As one last-ditch effort, I asked Hermes if there were any other models they would recommend from https://ollama.com/search, or do we pretty much already have what is the most optimal at the moment?

Ok, now I’m a bit confused about the Qwen versions. It recommended qwen3.6:27b and said that the 1 GB less memory for qwen3.5:27b wasn’t worth bothering with, since a newer version is out. Why did it skip over qwen3.8:27b, which is already downloaded and available on Ollama?

When asked, it stopped the background benchmark and started running a script against qwen3.8.

Well, I guess it didn’t realize that not only was a newer model out, but it was already downloaded and available to run locally. I thought it had already evaluated all available models on Ollama. Perhaps I should ask it to take a second look.

Hermes came back and confirmed that qwen3.8:latest was the 27B model and only 17 GB, leaving more VRAM headroom. Prompt processing for 57k tokens / 64k context exceeded the 420-second command-line limit, so it restarted the run as a background test. It would have been nice to try the 8k and 40k tests first so we could immediately see how it compares to the other models with those contexts.

Horrible, horrible… Just Horrible.

I can’t help but feel I’m being pushed toward a 3090 GPU to support newer NVIDIA drivers, features, speed, and the ability to run larger models using most of that card’s memory. I can’t help but wonder about many of the people who used to set up crypto mining rigs, which were mostly a bunch of video cards. Once mining went to Application-Specific Integrated Circuit (ASIC) miners and Field-Programmable Gate Arrays (FPGA), the video cards became too slow. They must be having a field day running large models on any hardware they have left over.

My guess is that these models will eventually go the same route as ASIC or FPGAs to get a reduction in power and an increase in speed. The problem is, models change fast, and building an ASIC or programming a FPGA runs into two problems – years of development, high costs, and lack of massive amounts of memory. I think the GPU’s meant for data centers (NVIDIA H100, Blackwell B200) may be a form of ASIC chips with tons of tensor cores for matrix multiplication compared to GPU’s for video games. It’s an expensive component that a hobbiest usually can’t afford.

Hermes came back with some results.

Laguna is rocking in speed compared to other models. I’m surprised at how badly Qwen 3.8 performs compared to its older coder model. I would have expected an improvement.

I’m done testing for the night. For now, the path forward is clear. Pick up 50% more work with my batch jobs and send it to the qwen3:8b model on the older 1080 Ti card. A batch that takes 47 days should take about 31 days on the older card. It’s not fast, but it can still significantly reduce the overall time to process everything.

A batch of 100 jobs was sharded, with GPU1 taking 71% and GPU2 taking 29%.

Looking at the initial benchmark tests. Context Generation was 110.7 vs 42.2 (72% vs 28%), and Prompt ingestion was 4,412.7 vs 726.5 (86% vs 14%). Asking Hermes to take on 50% more work in the batch processing jobs may have been a bit too aggressive.

The job sharding was reduced to 80% vs. 20%. So a 47-day job is reduced to 37 days. Not as good, but still 10 days faster than where I was this morning. I’ll take what I can get. That burn down will be quicker once I switch to a service that constantly feeds the models rather than cron jobs alone, with downtime if the job finishes early.

Hermes also asked me to set up a separate port for the web server to pin the GTX 1080 Ti card.

#!/usr/bin/env bash
# Install a second, localhost-only Ollama service
# pinned to the GTX 1080 Ti.
set -euo pipefail
install -d -m 0755 /etc/systemd/system
cat > /etc/systemd/system/ollama-gpu1.service <<'UNIT'
[Unit]
Description=Ollama Service (GTX 1080 Ti batch worker)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="OLLAMA_HOST=127.0.0.1:11435"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="CUDA_VISIBLE_DEVICES=1"
Environment="GGML_VK_VISIBLE_DEVICES=1"
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable --now ollama-gpu1.service
systemctl --no-pager --full status ollama-gpu1.service

And this is the file it created:

[Unit]
Description=Ollama Service (GTX 1080 Ti batch worker)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="OLLAMA_HOST=127.0.0.1:11435"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="CUDA_VISIBLE_DEVICES=1"
Environment="GGML_VK_VISIBLE_DEVICES=1"
[Install]
WantedBy=multi-user.target