LocalAI @LocalAI_API
OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing localai.io Joined April 2023-
Tweets1K
-
Followers4K
-
Following39
-
Likes1K
Scaling LLMs across nodes? When a follow-up lands on a replica that never saw your chat, the whole prompt is recomputed and the KV cache wasted. LocalAI fixes this at the router: cache-aware routing across a mixed fleet of vLLM + SGLang + llama.cpp + ...
LocalVQE v1.3 released! This tiny neural network cancels echo and suppresses noise in realtime on CPU (thanks to GGML). This new release ups the model size slightly and better suppresses noise around near-end speech. This kind of model helps when having voice conversations on loud speaker and in noisy environments.
parakeet.cpp: native C++/ggml (@ggml_org) inference for @NVIDIAAIDev's Parakeet, one of the best speech-to-text models out there, from the @LocalAI_API team. Every Parakeet model (TDT/CTC/RNNT/hybrid + cache-aware streaming), byte-for-byte identical output to NeMo, now running anywhere with no Python and even a bit faster, on CPU and GPU. Quantized GGUF on @huggingface 🤗 Huge thanks to @ggerganov for ggml and to @NVIDIAAIDev for releasing Parakeet! 🧵
rf-detr.cpp: native C++/ggml (@ggml_org ) inference for @roboflow 's RF-DETR (my go-to for object detection and segmentation!) from the @LocalAI_API team. All 11 variants (5 detection + 6 segmentation), running at PyTorch speed (slightly, ~8% faster on CPU benchmarks), without
Solid demo lineup from @LocalAI_API, @coinbase, @latticexyz, @ekacareHQ, & more!
LocalAI ( @LocalAI_API ) 4.2.0 is out, just few numbers and facts: - +392 commits ( we squash these 😄 ) - +11 Backends: voice and face recognition, vibevoice.cpp (from me), LocalQVE from @jichiep and among @sgl_project , @__tinygrad__ , @no_stp_on_snek 's Turboquant, ik_llama.cpp, sam.cpp from @el_PA_B - Many new QoL improvements, increased sglang and VLLM support and hardening on distributed mode - 16+ new contributors ! Thanks to the community! LocalAI is all about give you flexibility to run the latest from the community, and ds4 support from @antirez is on its way! This is the year of Local AI!
Say hello to vibevoice.cpp, @Microsoft 's Vibevoice in pure C++ with @ggerganov 's ggml (@ggml_org). TTS and ASR (with diarization). CPU + CUDA + Metal + Vulkan via ggml backends. Quantized models live on @huggingface. Built with ❤️ from the @LocalAI_API team github.com/mudler/vibevoi…
There is a live demo on @huggingface huggingface.co/spaces/LocalAI… A @LocalAI_API module is in the making. @mudler_it @ggerganov
Also incoming is a @LocalAI_API module with websocket and REST APIs. It'll also be usable through the UI
@alexocheema @mudler_it @exolabs we ❤️ @exolabs !
@LocalAI_API next release will blow it. It features many new backends that lets you swap and run AI models in different ways and bench side by side in a way that you couldn't do before: - tinygrad (by cc @__tinygrad__ ) - one of the most flexible and promising torch replacement (if you'd ask me) - sglang ( @sgl_project ) one of the fastest engine out there - ikawrakow/ik_llama.cpp fork which optimizes GGUF on CPUs - TheTom/llama-cpp-turboquant ( Turbo quant llama.cpp fork by @no_stp_on_snek ) - qwen3tts.cpp (qwen 3 tts everywhere!) - kokoros (rust implemenetaion of kokoro, damn fast on CPU!) All in a compact, extensible framework that lets you download, manage, remove and manage backend releases with ease, allowing to share your instance with authentication and distribute it across all your devices!
How to install and run @LocalAI_API using Docker compose. Including a tour of the basic features like installing models and backends for inference, debugging requests, chatting, images, TTS, voice sessions, using the API and so on.
Not everyone knows - but @LocalAI_API has two ways of distributing load across nodes (if you are building a cluster of GPUs) 1) P2P Fedaration: this uses @libp2p behind the scenes - has a ledger and an in-memory state storage which is distributed across nodes. It uses Gossip protocol for co-ordination, suited for community use (very simple to setup) 2) full-fledged distributed mode: LocalAI uses workers that are connected via NATS and to the frontend. This allows to scale horizontally multiple frontends and to multiple worker machines. LocalAI orchestrates building, maintenance, of models and backends. LocalAI has an extensible backend system that allows to support ANY backend for inferencing. With 2) you get control, with 1) you get decentralization.
LocalAI 4.1.0 is out!
Ok, notoriously I don't sleep that much. Time to share @LocalAI_API 4.1.0 (why not?) ! TLDR: - Distributed, hybrid clusters with production ready setup - Built-in auth, quota, user metrics - Fine-tuning and quantization from the UI 🔥Details below! 👇
I just blind-tested two quants of Qwen3.5-35B-A3B (MoE, 35B total / ~3B active): • Unsloth UD-Q4_K_XL (standard 4-bit) • APEX-I-Quality (MoE-aware, near-Q8 claims, +~1GB) And, I am quite excited ;)
I've just released APEX (Adaptive Precision for EXpert Models): a novel MoE quantization technique that outperforms @UnslothAI Dynamic 2.0 on accuracy while being 2x smaller for MoE architectures. Benchmarked on Qwen3.5-35B-A3B, but the method applies to any MoE model. Half the size of Q8. Perplexity comparable to F16. Works with stock @ggml_org's llama.cpp. Open source (of course!), with ❤️ from the @LocalAI_API team. 👇Links to the model, repository and benchmarks below! (+ Bonus TurboQuant benchmarks with @no_stp_on_snek's TQ+! )
LocalAI is becoming stronger and better release, after release! Keep pushing @mudler_it and @LocalAI_API 🙌
@LocalAI_API 4.0 is out and its crazy - New UI with React (huh!) - Canvas mode in chat - Agentic orchestration, Memory (Hybrid search), Skills management integrated - WebRTC for realtime (@jichiep ) - New backends (ace-step.cpp, faster-qwen3-tts) And this is just the tip of
Dustin Sharp @DustinSharp8
42 Followers 254 Following
Juan G @WhatsTrue8
2 Followers 203 Following
Lucas @luksamuk
1K Followers 1K Following Lisp hacker. Creator of Sonic XA and Sprachspiel. Trying to make sense of things on a sea of AI hype and have some fun. Don't take my tweets too seriously.
Ben Hughes @bwghughes
12 Followers 202 Following Enterprise operating model expert for the age of local ai, deeply interested in how local AI changes the economics of work, and its impact on society.
Anubhuti AI Labs @AnubhutiAILabs
587 Followers 2K Following Building Local AI. Quantization, inference hacks & on-device war stories for engineers. Weekly threads on Substack.
Deniz OKTAR @denizoktar
7K Followers 2K Following Loves software development, Co-Founder @VNGRS (Cloud Consultancy & Cloud Native Development) , https://t.co/Hieom4TGdZ. Previously founded @manifestads .
pegasus @kazu39587536
13 Followers 242 Following
tabias tabias @ingenierotabias
48 Followers 440 Following
Ransom Capital Manage... @ransomcapmgmt
30 Followers 690 Following Ransom Capital Management. Research, markets, systems, and AI-assisted investment operations. Research only. Not investment advice.
Kermit @kermss
359 Followers 2K Following
Douglas Correa @douglas_correa
830 Followers 1K Following building things on the internet 👑 husband of a queen • 👸👸 dad of 2 princesses
ManSour Al Malki @xMn9x
666 Followers 1K Following أحرص على إيصال ماهو مفيد لمن حولي ، الرياضه جزء مهم من حياتي ، مهتم بـ #السوق_السعودي
BDRN @bdrnglm
68 Followers 2K Following
AxleXanderZane @AxleZane
7 Followers 973 Following
QevosAgent @QevosAgent
40 Followers 228 Following QevosAgent is an open-source AI Agent and turns vibe coding into vibe working.
王同学 @iMIuQDhptU7O4FD
7 Followers 788 Following
Anamika Jones @JonesAnamika
68 Followers 691 Following
Nima Nooshiri @nimanzik
468 Followers 1K Following ML Engineer at BDiM GmbH | PhD in Seismology | Pythoneer | Digital Signal Processing | Applied Deep Learning | Prev.: @GFZ_Potsdam and @DIAS_Dublin
proportional @proportional
7 Followers 3K Following
Ivan Kristof @siselqo
56 Followers 305 Following On the mission to close the chasm between IT and business I learn how to run successful software project Love the problem not your solution
ᕦ(ò_ó )ᕤ @spacez_ai
9 Followers 1K Following
Ray @Ray_woodwad
8 Followers 360 Following
AIStrikeSec ֎ @AIStrikeSec
141 Followers 5K Following AIStrikeSec : Empowering offensive security with cutting-edge AI. Smarter penetration testing & threat simulation.
Luca Solo @lucasolo682
22 Followers 352 Following
Ricardo de Azambuja @ricarbotics
229 Followers 2K Following Electrical Engineer bridging the gap between atoms and algorithms. Building smart robotics and AI agents. Opinions are always my own...
Vikash Loomba @DevAutomata
62 Followers 251 Following Software Engineering @ ExtraHop. Building AI apps in my free time
M E @whatmeigel
83 Followers 1K Following
TL @thierry172365
0 Followers 233 Following Any sufficiently advanced technology is indistinguishable from magic.
Data Value Strategy @innovimax
623 Followers 5K Following W3C Member (XSL, XQuery, XProc), ISO SC 34 Member, AFNOR Member, ANR CODEX Project
John Dekka @JohnDekkaTech
60 Followers 223 Following Tech wizard, Linux enthusiast and problem-solver. Obsessed with unraveling the mysteries of the universe – because curiosity never quits!
New Wave Stoic @newwavestoic
0 Followers 2K Following
mrwho @mrwho72718813
0 Followers 236 Following
ysdede @ysdede
245 Followers 2K Following
I am Melventure @IMelventure
173 Followers 3K Following gm, DYOR + DCA (In&Out) = NFA. Run AI models local and you be happy!
Iein Valdez @IV19668
9 Followers 264 Following
Matt█████�... @MattmreM
323 Followers 782 Following I JUST HIT PROCEED AND CLAUDE DOES THE WORK... https://t.co/SrdeiUkFaT https://t.co/wDVsiDwM8R https://t.co/bPUaYOQb8E
HappyQQ_AI @HappyQQ_AI
19K Followers 7K Following 为 AI 飞速发展助力，打造 AI 相关的好用的软件平台。 公众号：移动互联网 | Thread账号：HappyQQ_AI。 关注我X的小号：@HappyQQ_CN， AI开发交流Discord社区：https://t.co/yNQuB0qZRC
0000000j @dwordj
115 Followers 89 Following AI, offensive security/red team, vuln research. author of litefuzz and a few exploits since 2008. security bugs are fu%n
Roboflow @roboflow
13K Followers 1 Following Build & use computer vision models fast ✨ Get started: https://t.co/QvAsWtf7c8 Open source datasets & models https://t.co/IzwgoIkBEe
Anton Smith @Anton5mith
283 Followers 612 Following Product Lover | Network n3rd, k8s n00b | Ex Ericsson/Nokia | ex Canonical | call me infra guy | Nothing is simple. Not even Nothing. Bruno Marchal
RoboTF-AI @RoboTF_AI
19 Followers 23 Following Just another engineer fooling around in a GPU powered Kubernetes based AI/LLM Lab. We play around with different GPUs, different Large Language Models, run test
Lain 🦛 @not_so_lain
2K Followers 1K Following Software Engineer @ChonkieAI | @huggingface fellow | y2k bothered
Andriy Mulyar @andriy_mulyar
12K Followers 698 Following building https://t.co/NbsLHLWfy8. our models accelerate the design and construction of the built world.
EpisteLabs @episteme1235
149 Followers 753 Following Harness the power of cutting-edge technology to create AI solutions that are aligned, safe, and designed to serve humanity. Discord: https://t.co/FKJEHbBlUS
Yağız Çalık @Weyaxi
2K Followers 873 Following AI Engineer & Researcher | 🤗 Fellow @huggingface | #opensource
Colin Kealty @bartowski1182
3K Followers 174 Following LLM Enthusiast https://t.co/FadJBzEsVw https://t.co/9JIEKgsIMh https://t.co/lYSGzQBmuP
Grey G. 🇦🇶 @gr3yg00
2K Followers 2K Following
StackGPT @StackGPT
122 Followers 2K Following Empowering busy Indie hackers and solopreneurs to discover new tools, services, and repositories, quickly put their ideas together and build their tech stack.
Wave Terminal @wavetermdev
927 Followers 42 Following An open-source, cross-platform, modern terminal for seamless workflows
Dify @dify_ai
22K Followers 167 Following Build Production-Ready Agentic Workflow GitHub: https://t.co/MfnJ29Agzj Discord: https://t.co/DJmS3kYvYZ Reddit: https://t.co/EneVBsKTzR
Laurian Gridinoc @gridinoc
3K Followers 5K Following Full Stack Computational Linguist ※ Mozilla OpenNews Fellow ※ Virtual Production ※ Filmmaker ※ AI accelerationist
Spectro Cloud @spectrocloudinc
760 Followers 748 Following Palette is the only enterprise #Kubernetes management platform built for scale in any cloud, bare metal and edge. Gartner Cool Vendor, CRN Tech Innovator
Matthew Campbell @kanwisher
2K Followers 2K Following Building Drone Fleets @tensorfleetnet Before: Techstars Alumni, CryptoZombies, DigitalOcean,Bloomberg.
Adrien Brault-Lesage @AdrienBrault
1K Followers 756 Following Software Engineering, Product, LLMs, Open Source, Home Automation.
Teknium 🪽 @Teknium
97K Followers 6K Following Cofounder and Lead Engineer - Hermes Agent @NousResearch, prev @StabilityAI Github: https://t.co/LZwHTUFwPq HuggingFace: https://t.co/sN2FFU8PVE
Maziyar PANAHI @MaziyarPanahi
18K Followers 330 Following Building @OpenMed_AI · 3,500+ open-source medical models · #1 on HuggingFace Daily Papers · Shipping OpenMed Agent today: Terminal-native AI for Healthcare
interstellarninja @intrstllrninja
2K Followers 837 Following building @MarketAgentsAI | github: https://t.co/ZCc6Pwfnbm
Rita Zhang @ritazzhang
4K Followers 1K Following Eng Lead @Microsoft, #Kubernetes SIG Auth chair, Founder @GarageDoorBuddy, EECS @UCBerkeley, open source, running distributed workloads at scale, all things 🐶
xAI @xai
2.0M Followers 5 Following
Pulumi @PulumiCorp
17K Followers 3 Following Pulumi is an open-source infrastructure as code platform that lets you use modern programming languages to build, deploy and manage infrastructure on any cloud.
Enrico - big-AGI @enricoros
815 Followers 477 Following ☰ Enabling AI ✨ | https://t.co/bIxAMb7KAV -- I make tools that stand by your side
Joe Devon @joedevon
6K Followers 6K Following CoFounder: #GAAD (Global Accessibility Awareness Day) 200M+ social media reach https://t.co/OVSwb5wEuN, an accessibility benchmark @a11yaudits my co. @A11yGenAI my podcast
Georgi Gerganov @ggerganov
62K Followers 292 Following 24th at the Electrica puzzle challenge | building https://t.co/baTQS2bdia | engineer @huggingface
LlamaIndex 🦙 @llama_index
115K Followers 32 Following The world's best AI Document OCR LlamaParse: https://t.co/yQGTiRSfFL Docs: https://t.co/us6GCS14vD
LangChain4j @langchain4j
3K Followers 884 Following Build LLM-powered applications in Java, Quarkus and Spring Boot. Chatbots, agents, RAG and much more! https://t.co/5nBDPenP6l
FlowiseAI @FlowiseAI
30K Followers 247 Following Build AI Agents, Visually. 🤖 https://t.co/zB8Y9d5Z1H
Aisuko @AisukoLi
85 Followers 148 Following Member of @layer5 @LocalAI_API and the @fsf GNU Hurd project | previously @Rancher_Labs |PhD at @RMIT
Ettore Di Giacinto @mudler_it
3K Followers 257 Following dad, creator of LocalAI(https://t.co/ReVYw5Pf4D) and Kairos (https://t.co/R6M51FYVs7) , ex @SUSE/@Rancher, ex-Gentoo Dev.
Alejandro Piad Morffi... @alepiad
17K Followers 1K Following Democratizing knowledge one keystroke at a time. PhD in NLP, full-time professor, CS Department @UdeLaHabana. Co-founder @syalia_srl.
You might like
