* Macbook is not an option
* I go through phases and switch between Windows and Linux as my primary OS.
* Want to be able to mess around with some local LLMs.
* I travel frequently, so portability is somewhat important. Currently own a 13 inch, but 14 inch should work too I think.
Notes:
* My current laptop is a 6yr old Dell XPS. It has generally served me well.
* I bought an Asus Zenbook for a family member and I have been impressed with how well it has worked out. Anyone with any recent experience with Asus laptops for development?
* I have had bad experiences with Lenovo twice, which makes me wary of Thinkpads, but willing to consider it if it makes the most sense.
* Framework look very appealing but I have heard mixed reviews.
* Macbook is not an option
* Want to be able to mess around with some local LLMs.
Your choices for a Window laptop that can run a local LLM is either to get a large amount of system RAM and have it be abysmally slow, or to run a very tiny model on a discrete GPU which will (a) not be very good due to its small size and high quantization and (b) evaporate your battery life.
If you want to run local LLMs on a laptop and actually have them be useful, a Mac is currently the only real choice.
That said, with the money you save buying a Linux laptop instead, you can pay for a lot of tokens for whatever hosted LLM you want and it will be higher quality than what you could potentially run locally on a Mac.
Specs aside, this machine can also run Linux well [1], which makes it worth buying, IMO. And if it can run Linux well, then using it for development should be a breeze.
I agree that MacBook is not an option. I have a work MacBook and it has better silicon, ok, but I hate being on a Mac. Just not for me.
I'd wait until more manufacturers announce what they will do with that AMD SOC.
- Framework reviews are important to figure out the current generation. I tried one out and it had few issues other than me not having the time to tweak it how I wanted. Hardware was great, battery wasn't great at that time (needed simple software optimizations), but Framework now has larger batteries too that seem to have resolved that. The folks I know with them have mostly switched from Mac or X1 Carbons to Frameworks and quite happy with it.
- Specifically for travel, the recently announced Framework 12" looks really inviting if travel looks like the way to go.
To run an LLM on either, some amount of extra ram might help, depending on the model... if you want something heavier it might be cheaper to run a model privately in the cloud as needed, or use an eGPU and plug it in when you want. Local LLM use seems to be fun, but having something at home running it so it's accessible seems serviceable too.
It's not cheap and rather heavy, but very powerful. You can get it with lots of RAM, a powerful graphics card (for LLMs), and a CPU with 24 cores (32 threads). And you can get it shipped with Fedora or Ubuntu (although wiped it immediately and installed my own Fedora).
Curious what the bad Lenovo experience was. I've had various Thinkpads since the early 2000's and they all worked for me.
Since I'm in the EU, I ended up with 2 possible options: ThinkPad T14 (non-s model), Gen.5 or newer. Or Framework 13 AMD.
System76 also make some interesting laptops, but it's just too expensive to get one in Europe.
Your backend box can be as ugly as you like whilst your front end laptop is good enough for normal usage.
Quantized LLMs upto 8b parameters can be easily run on above specs. Quantized models are getting better and better. I use them for code suggestion and code gen.
Instead of Dual Boot they also offer a preinstalled VM with a licensed Windows. Tuxedo OS Linux is a slightly modified Ubuntu optimized for their Hardware.
- 45W TDP processor - none of the U series 15W slowness
- 4K screen that scaled to 1080P perfectly at 200% scaling - so X11 and XWayland apps still looked good. Also the screen quality was amazing
- Integrated Graphics - no messing around with NVIDIA drivers
I searched high and low for something similar but could not find anything on the market. So made some compromises and went with System76 Pangolin (pang15). Here are my notes on the machine Pros:
- 45W TDP processor is fast
- They let you load it up with memory and disk for reasonable prices unlike most mainstream manufacturers
- 2nd M2 slot is nice for copying over old SSD
- They provide Windows Drivers in a GitHub repo
Cons:
- Battery life is pretty bad but I stay plugged in most of the time so not a huge issue for me
- Fans spin up pretty often
- USB-C charging only. There is an A/C Adapter barrel jack but they don't ship a charger and I can't find a suitable aftermarket charger. I worry about wear and tear on the USB-C port damaging it one day and that will be the end of the laptop effectively if I can't plug in.
- The screen needs around 133% fractional scaling which is not even an option. So I have to use a combination of 125% scaling and tweaks. XWayland apps look terrible so had to go through and force Wayland on all Electron apps and JetBrains and as you can probably guess there are random odd bugs
- Random issue (rarely) where I power it on and it does not start up, then I have to close the lid, hold the power button, etc before it starts up. Support told me to try booting with one memory stick but it's so hard to recreate I don't think I'll be messing around with it unless the issue starts cropping up more.
Now given all of the complaints, the increased speed over a 7-year old laptop is still nice so I am keeping it. Too bad Dell seemingly abandoned their Developer Edition program. The new XPS's force discrete graphics for good screen options and it's a roll of the dice whether WiFi or peripherals work properly with Linux. I've heard Lenovo has good Linux support for certain things but they also seem to be pushing discrete graphics for high screen resolutions.
Franework is nice but no Nvidia option which is a no-go for LLM.
Thinkpad P1 Gen 6 with RTX 4090 is a solid one. Older generation but Thinkpad seems like no longer slapping top GPU into its notebook.
Other laptops with 4090/5090 are too huge (Titan, Raider, Vector, Strix Scar 18,…)
I’m using Arch on the Thinkpad above.
Hope this helps
I was just researching this for my own use.
Running LLMs on a local mobile device doesn't make sense to me. Sure it sounds great but at what cost & what trade-offs? This goes for laptops & phones.
You can have the best of both worlds by setting up a separate computer in your local network or rent one on the cloud if data privacy & messing around with LLMs is important to you. Then you don't sacrifice the mobile advantage of a great laptop. It'll be cheaper & can be more focused too.
Thinkpads are reliable and great QA/support, but the HW is underwhelming, bad thermals, dim screens, and just too expensive.