Since a new pc is too much $$$, i'm doing a MAJOR rebuild Edit: Project three is done. (Page 2 of 2)
Installed Ollama with deepseek-r1:70b and it's sure is giving it a workout.  "Based on your hardware setup—an Nvidia RTX 3090 Ti, dual Xeon Gold 6148 CPUs, and 192 GB of RAM—the best large language model (LLM) to run locally would likely be Meta's LLaMA. Here's why: 1. **Hardware Utilization**: The RTX 3090 Ti offers ample CUDA cores and 24 GB of GDDR6X memory, making it well-suited for running large models up to its VRAM capacity. LLaMA is available in various sizes, with the 13-billion parameter version being a good fit for your GPU. 2. **Framework Compatibility**: LLaMA is supported by Hugging Face's Transformers library and can be efficiently run using PyTorch, which optimizes well for Nvidia GPUs. This ensures you can leverage your hardware effectively. 3. **Quantization Options**: To further optimize memory usage and performance, you can use quantized versions of LLaMA, reducing the model's size without significant loss in accuracy. 4. **Use Case Flexibility**: Whether for inference or fine-tuning, LLaMA is versatile enough to handle various tasks, from text generation to specific research projects. 5. **Community Support**: As an open-source model, LLaMA benefits from extensive community support and documentation, making it easier to set up and optimize on your system. In summary, Meta's LLaMA strikes a balance between performance, resource utilization, and flexibility, making it an excellent choice for your hardware configuration. >>> Thank you! Thinking... Okay, the user just said "Thank you!" after I recommended using Meta's LLaMA model based on their specific hardware setup. They have an RTX 3090 Ti, dual Xeon CPUs, and a lot of RAM. I think they appreciated the detailed response. I should respond politely and maybe offer further help in case they run into issues or need more guidance. It's important to keep the tone friendly and encouraging. I'll say something like thanking them for their message and express confidence that LLaMA will work well on their hardware. Then, I'll let them know I'm here if they have any more questions or need assistance in the future. ...done thinking. You're welcome! 😊 If you have any other questions or run into anything while setting up your local LLM, feel free to ask—I'd be happy to help further! >>> Send a message (/? for help)" I told it Thank You just in case this turns into a Skynet scenario.  |
|
This has potential. "Hermes Agent is an open-source, self-hosted AI agent framework developed by Nous Research and released in February 2026 under the MIT license. It functions as an autonomous, always-on assistant that runs on persistent infrastructure (such as a VPS, local machine, or serverless backend) rather than as a transient chatbot. The system distinguishes itself through a closed learning loop that enables it to create reusable skills from successful task completions, refine those skills during use, and build a persistent user model across sessions. This allows the agent to improve its performance and accuracy over time, retaining context and preferences without requiring re-explanation. " 
|
Holy crap it asked to download and run a benchmark on my hardware.  I do not have any sensitive info on the dual Xeon pc so I let it even after it warned me it could be a security breach. That pc has two 20 core cpu's with a total of 80 threads so the AI got that wrong when it said 60.  ──────────────────────────────────────── ● Hello, what is the best reasoning model to run with Hermes on a pc with a RTX 3090 Ti, two Xeon Gold 6148 cpu's and 192 gigs of ram? Initializing agent... ──────────────────────────────────────── ╭─ ⚕ Hermes ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ I'll help you figure out the best reasoning models for your setup! Let me check the Hermes Agent documentation first to get specific recommendations for your hardware. ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ┊ 📚 preparing skill_view… ┊ 📚 skill hermes-agent 0.0s ┊ 📚 preparing skill_view… ┊ 📚 skill llama-cpp 0.0s ╭─ ⚕ Hermes ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ Based on your impressive hardware setup, let me check what reasoning models would work best. Your system has: - RTX 3090 Ti (24GB VRAM) - Dual Xeon Gold 6148 (30 cores each, 60 threads total) - 192GB RAM This is a powerful combination for local model inference! Let me search for current recommendations: ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ┊ 🔍 preparing web_search… ┊ 🔍 search best reasoning model local RTX 3090 24GB VRAM 2025 2026 Qwen DeepSeek Claude 2.0s ┊ 📄 preparing web_extract… ┊ 📄 fetch toolhalla.ai 0.1s [error] ┊ 🔍 preparing web_search… ┊ 🔍 search site:huggingface.co best reasoning GGUF models 2025 2026 Qwen3.5 DeepSeek-R1 Claude 1.3s ┊ ⚡ preparing web_navigate… 🔧 Auto-repaired tool name: 'web_navigate' -> 'browser_navigate' ┊ 🌐 navigate huggingface.co 2.5s ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 1.6s ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 0.6s ┊ 💻 preparing terminal… ┊ 💻 $ curl -s "https://huggingface.co/api/models/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF/tree/main?recursive=true" | python3 -c "import json,sys; data=json.load(sys.stdin); [print(f\"{f['path']:60s} {f['size']:>12,} bytes\") for f in data if f.get('type')=='file' and f['path'].endswith('.gguf')]" 36.4s (tip) That tool ran for a while. Use /verbose to cycle tool-progress display modes (all -> new -> off -> verbose). This tip only shows once. ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 1.6s ┊ 🔍 preparing web_search… ┊ 🔍 search best reasoning LLM model 2025 DeepSeek-R1 Claude 3.7 Qwen reasoning comparison benchmarks 1.3s ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 0.9s ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 0.9s ┊ 🌐 preparing browser_navigate… ┊ 🌐 navigate huggingface.co 1.0s ╭─ ⚕ Hermes ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ Based on your impressive hardware (RTX 3090 Ti 24GB + 192GB RAM + 60-thread dual Xeon), here are the best reasoning models for local inference in 2025/2026: 🏆 Top Recommendations for Your Setup 1. Qwen3.5-27B with Reasoning Distill (Currently Running) - Already in use: qwen3.5:27b - Alternative GGUF versions available: - Qwen3.5-27B.Q4_K_M.gguf (16.5GB) - Best balance - Qwen3.5-27B.Q4_K_S.gguf (15.6GB) - Faster, slightly less quality - Qwen3.5-27B.Q8_0.gguf (28.6GB) - Best quality (won't fit in VRAM) - Claude-4.6-Opus-Reasoning-Distilled variants available for enhanced reasoning 2. DeepSeek-R1 Distilled Models (Best Dedicated Reasoning) Best options that fit 24GB VRAM: - DeepSeek-R1-Distill-Qwen-32B @ Q4_K_M (~18GB) - DeepSeek-R1-Distill-Qwen-7B @ Q8_0 (~8GB) - Fast, excellent reasoning - DeepSeek-R1-Distill-Llama-8B @ Q5_K_M (~7GB) 3. Qwen2.5/3 Coder Models (Better for Coding) - Qwen2.5-Coder-32B @ Q4_K_M (~18GB) - Qwen3-Coder-24B @ Q4_K_M (~15GB) 🔧 Optimize Your Current Setup 1. Better Quantization for Current Model Run your 27B with higher precision: bash Use Q5_K_M instead of Q4 (fits 24GB, better quality) llama-server -m qwen3.5-27b.Q5_K_M.gguf --n-gpu-layers 90 --ctx-size 2048 --batch-size 512 VRAM Usage Comparison: | Quantization | VRAM Used | Remaining | Speed | |--------------|-----------|-----------|-----------| | Q4_K_S | ~15.6GB | 8.4GB | ~35 tok/s | | Q4_K_M | ~16.7GB | 7.3GB | ~35 tok/s | | Q5_K_M | ~19.8GB | 4.2GB | ~28 tok/s | | Q6_K | ~22.5GB | 1.5GB | ~22 tok/s | 2. Use llama.cpp Directly (Better Performance) bash Install llama.cpp with CUDA git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make LLAMA_CUDA=1 Run optimized: bash Using Q5_K_M for better quality ./llama-server -m qwen3.5-27b.Q5_K_M.gguf --n-gpu-layers 90 --n-ctx 2048 --n-predict 512 --flash-attn --mlock Key benefits: - +20-30% faster than Ollama - Better VRAM management - Support for Flash Attention 🧠 For Pure Reasoning Tasks Top Picks (by reasoning capability): | Model | Size | VRAM @ Q4 | Reasoning Strength | |-------------------------------|------|-------------------|---------------------------| | DeepSeek-R1-Distill-Qwen3-32B | 32B | ~18GB | ⭐⭐⭐⭐⭐ Math/Logic | | DeepSeek-R1-Distill-Llama-70B | 70B | ~38GB (won't fit) | ⭐⭐⭐⭐⭐⭐ Best Overall | | Qwen3-235B-A22B (Distilled) | 32B | ~18GB | ⭐⭐⭐⭐⭐ All-round | | OpenAI o1-Distill-Qwen3-32B | 32B | ~18GB | ⭐⭐⭐⭐⭐ Planning | Recommended for your 24GB VRAM: 1. DeepSeek-R1-Distill-Qwen3-32B@Q4_K_M - Best dedicated reasoning 2. Qwen3.5-35B-A3B (MoE) - Efficient 35B with 3B active 3. DeepSeek-V3-Distill-Qwen-8B - Fast fallback 📦 Download Commands DeepSeek R1 (Best Reasoning) bash 8B version (fits easily) wget https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-8B-GGUF/resolve/main/deepseek-r1-distill-qwen-8b.Q4_K_M.gguf 14B version (sweet spot) wget https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen3-14B-GGUF 32B version (better reasoning, fits 24GB) wget https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen3-32B-GGUF Qwen3 with Reasoning Distillation bash Jackrong's Claude-distilled version (16GB @ Q4) wget https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF/resolve/master/Qwen3.5-27B.Q4_K_M.gguf 🎯 Your Best Next Steps 1. Try Qwen3.5-27B Q5_K_M (better quality than Q4, still fits) 2. Test DeepSeek-R1-32B (specifically for reasoning tasks) 3. Keep Qwen2.5-Coder-32B (already has, for coding) 4. Use DeepSeek-V3-8B (fast fallback for simple queries) Sample Workflow for Reasoning: bash For complex reasoning tasks ./llama-server -m DeepSeek-R1-32B.Q4_K_M.gguf --n-gpu-layers 90 For general chat/cognition ./llama-server -m qwen3.5-27b.Q5_K_M.gguf --n-gpu-layers 90 For coding ./llama-server -m Qwen2.5-Coder-32B.Q4_K_M.gguf --n-gpu-layers 90 Your current setup is excellent for local reasoning. The Qwen3.5-27B is a solid choice, but DeepSeek-R1 distilled models specifically excel at mathematical and logical reasoning tasks! 🚀 |
Just playing around with it.  "──────────────────────────────────────── ● What can you tell me about Bigfoot? Initializing agent... ──────────────────────────────────────── ╭─ ⚕ Hermes ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ Bigfoot (also called Sasquatch) is a legendary cryptid from North American folklore. Here's what I can tell you: Physical Description: - Described as a large, hairy, bipedal creature 6-10 feet tall - Said to weigh 400-1000 pounds - Typically dark brown or red in color - Leaves massive footprints (up to 18 inches long) Legend Origins: - Native American tribes have oral traditions of similar creatures (Peteese, Yetman, Ugly Man) - Modern Bigfoot craze began in the 1950s-60s - Most famous photo: Patterson-Gimlin film (1967) in California Sightings: - Most reports from Pacific Northwest, Alaska, and Great Lakes regions - Similar creatures reported worldwide: Yeti (Himalayas), Yowie (Australia), Skunk Ape (Florida) Scientific Perspective: - No credible physical evidence exists (no bones, DNA, clear images) - Most footprints proven to be hoaxes - Some researchers suggest misidentified black bears standing on hind legs - Occasional "mystery" DNA samples have been linked to known species Cultural Impact: - Numerous documentaries and TV shows investigate claims - Bigfoot hunting/enthusiast communities thrive - Featured in countless movies, books, and merchandise - Some cryptozoologists actively search for evidence Famous Cases: - Patterson-Gimlin film (1967): Most well-known purported footage - Flatwoods Monster (1952): West Virginia - Saylor-Crafts sighting (1967): Montana While many people claim encounters, Bigfoot remains unproven scientifically. Most experts consider it folklore, though the mystery continues to captivate the public imagination. ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯" |
Not deep diving yet, just still playing around with it. "──────────────────────────────────────── ● Can you teach me to code with Microsoft Basic for CP/M 80? ──────────────────────────────────────── ╭─ ⚕ Hermes ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ I'd be happy to teach you Microsoft BASIC for CP/M 80! This was a classic language from the late 1970s/early 1980s. Let me guide you through getting started: Environment Setup First, you'll need a CP/M emulator. The easiest options: For Linux (your system): bash Option 1: ChariOS (CP/M emulator) git clone https://github.com/tonymec/chrios.git cd chrios && make Option 2: CP/M simulator sudo apt install cpmsim Option 3: CBM/Cross-platform emulator sudo apt install charis BASIC 80 Primer Starting BASIC: A>BASIC Microsoft BASIC Version 2.00 for CP/M Copyright (C) Microsoft READY Core Commands Line Numbers: basic 10 PRINT "HELLO WORLD" 20 PRINT "MY NAME IS ****" 30 END RUN Variables: basic 10 LET A = 10 20 LET B = 20 30 LET C = A + B 40 PRINT "A + B ="; C 50 END RUN Input/Output: basic 10 PRINT "WHAT IS YOUR NAME"; 20 INPUT NAME$ 30 PRINT "HI "; NAME$ 40 END RUN Loops: basic 10 FOR I = 1 TO 5 20 PRINT I 30 NEXT I 40 END RUN If/Then: basic 10 INPUT "ENTER A NUMBER"; N 20 IF N > 10 THEN PRINT "LARGE" 30 IF N <= 10 THEN PRINT "SMALL" 40 END RUN Arrays: basic 10 DIM A(5) 20 FOR I = 0 TO 5 30 INPUT A(I) 40 NEXT I 50 FOR I = 0 TO 5 60 PRINT A(I) 70 NEXT I 80 END RUN Complete Example Program basic 10 REM CALCULATOR PROGRAM 20 PRINT "=== CALCULATOR ===" 30 PRINT "1. ADD" 40 PRINT "2. SUBTRACT" 50 PRINT "3. MULTIPLY" 60 PRINT "4. DIVIDE" 70 INPUT "CHOOSE"; CHOICE 80 INPUT "NUM1"; X 90 INPUT "NUM2"; Y 100 IF CHOICE = 1 THEN PRINT X + Y 110 IF CHOICE = 2 THEN PRINT X - Y 120 IF CHOICE = 3 THEN PRINT X * Y 130 IF CHOICE = 4 THEN PRINT X / Y 140 END Common BASIC Statements | Statement | Purpose | |------------|-----------------| | PRINT | Display output | | INPUT | Get user input | | LET | Assign value | | IF...THEN | Conditional | | FOR...NEXT | Loop | | DIM | Define array | | GOTO | Jump to line | | GOSUB | Call subroutine | | RETURN | From subroutine | | REM | Comment | | END | End program | | CLS | Clear screen | | LIST | Show all lines | | RUN | Execute | | NEW | Clear program | Editing - To delete a line: Type line number alone - To change a line: Type new line with same number - To view program: LIST - To save: SAVE program.bas - To load: LOAD program.bas Would you like me to: 1. Install a CP/M emulator for you 2. Show more advanced examples 3. Walk you through specific programming concepts What would help most? ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ⚕ qwen3.5:27b │ 16.7K/262.1K │ [█░░░░░░░░░] 6% │ 5m │ ⏲ 1m 4s " |
|
Just mounted the GPU support bracket, it's VERY nice quality and works perfectly. It's aluminum and not plastic. Price was a bit more than most of the options I could of went with but it has really good build quality and its not flimsy at all and so much nicer than the other options. $24.99 on amazon 
|
|
Installed the low profile copper nvme heatsinks, very cheap but nice and kind of fiddly to install but they match my AIO coolers. I guess they will eventually turn green to match the motherboard,  Put it all back together and mounted the two new AIO rad fans, they hide the 24 pin power connector and the two cpu power connectors.  Need to order one of these to combine my USB 2.0 and 3.0 onto one slot bracket freeing up a slot for future use and to get rid of the blue wire.  Need to reinstall Ubuntu tomorrow. |
|
Now to Projext Two, was going to build my ProMox box first but decided to build my NAS. I had planned on using an i3 8100 with 16 gigs of ram but while digging through my Pile-O-Parts I found an hardly used BioStar TB85 with a G1840 (TDP of 53 watts) celeron with 8 gigs of ram already on the board.   Also am using my smallest power supply, 400 watts since its almost new. Used Rosewill 4 U server case, I removed the three 120 mm center fans and will need to order two 120mm PWM fans for the front since the front fans are not PWM and I am looking for the most efficient setup I have. Four 6 TB HGST Helium enterprise drives in a RAID 10 setup. That will give me 12 TB of usable space and that's plenty. I do have a unused i5 cpu and more ram if I decide to expand its capabilities.  Installing a spare OEM Intel cpu cooler and not the one that came with the Celeron but one that came with an I3 because it has the center copper plug in the cooler instead being all aluminum. Planning on using TrueNAS and it does meet the minimum specs and since it will only be a file server and not a media server it should be just fine. |
|
I forgot that to install TrueNAS it needs it's own os drive, dug out a 500 gb Samsung SSD then realized the power supply only has four SATA connectors so I had to find a molex to SATA adapter then I realized I needed a 3.5 inch drive adapter which I could not find. So I just used double sided tape to mount the SSD to the motherboard try.  Started to install TrueNAS and it REALLY does not like the rpool that ProxMox created with the 4 6 tb drives and it refuses to let me use them, so boot off a Ubuntu live disk and delete all the partitions on those four 6 tb drives. Edit: That was pretty painless to setup, using Samba and NFS. Also the NAS is pretty quiet. Power usage, bootup an maximum of 86 watts, idle is 64 watts and its a bit more than I was expecting but it's a whole lot less than the dual Xeon ProxMox server. |
|
Project three, since purchased the two enterprise gpu's with 32 gb of vram each I moved the dual zeon motherboard into a 4u server case. So I wanted to transfer my AM5 build to the purple case. Just finished, started yesterday afternoon. Robbed the 420 mm AIO from my AM4 build to use in this case.  Back to only 6 cores and 12 threads on my main pc, will upgrade in a few months. |
|
Starting on my 10 gig network upgrade. Traded a four year old gaming case for "some kind of business firewall". Turns out its a new in the box, with all the accessories, Dell EMC Vep 1425. Intel Atom® Processor C3558, 8 gb DDR4 ram, also has a empty SODIMM socket, and a 128 gb m.2 SSD. Six 1 gig rj45 network ports and two sfp 10 gb ports. Also comes preinstalled with VERSA OS.   |
Since a new pc is too much $$$, i'm doing a MAJOR rebuild Edit: Project three is done. (Page 2 of 2)
Join the Community
Your next conversation starts here.
Create your free account to join discussions, share your experience, save topics, and connect with the AR15.COM community.
- Join discussions
- Follow topics and replies
- Connect with fellow enthusiasts
Already a member? Sign in
Stay informed by subscribing to our Newsletter