ModelVault is building the worldâs first fully-integrated, on-prem AI appliance â a plug-and-play AI server preloaded with software, models, and tools that let businesses (and eventually consumers) run powerful AI models locally, privately, and securely.
We're looking for a Senior Software Engineer to help us build the core developer experience and control plane for our on-prem systems â everything from containerized runtime orchestration to local APIs, CLI tools, OTA update flows, and lightweight UIs. You'll be one of the first engineers on the team and play a foundational role in shaping our system architecture, tooling, and technical culture.
ð What Youâll Do
- Design and build the software layer that makes ModelVault âjust workâ â including:
- Local APIs and system services
- Developer-facing CLI tools
- Simple admin web UI or dashboards
- OTA update infrastructure
- Wrap system-level functionality into intuitive interfaces for local users (e.g., âstart model,â âview logs,â âupdate softwareâ)
- Integrate and manage containerized LLMs and vision models (e.g. Llama 3, Mistral, SDXL) for local inference
- Collaborate with our systems and hardware engineers to optimize performance across the full stack â from GPU runtimes to dev UX
- Participate in architecture decisions related to system orchestration, hardware/software coordination, and secure update delivery
- Interface directly with early users to gather feedback, debug edge cases, and iterate quickly
- Contribute to hiring and scaling the engineering team over the next 12 months.
ð§ You Might Be a Fit If You
- Have 5+ years of software engineering experience â ideally in devtools, infra, or platform teams
- Are comfortable working close to the system â building tools that wrap around Docker, shell scripts, or system services
- Enjoy working across the stack â from lightweight frontend dashboards to backend APIs to scripting hardware operations
- Have experience building interfaces for technical users (CLIs, admin panels, config systems)
- Have worked with containerized environments and understand Docker, Compose, systemd, etc.
- Like solving ambiguous technical problems with minimal abstraction â and high leverage
âï¸ Bonus Points
- Built tooling or platforms for AI workloads (e.g. inference runners, LLM wrappers, logging for local models)
- Experience with GPU systems, Nvidia drivers, CUDA, or model performance tuning
- Familiarity with update mechanisms (auto-updaters, patch delivery, rollback safety)
- Past experience with embedded systems, local-first products, or edge computing
âï¸ Our Tech Stack (Currently)
- Python, FastAPI, Shell scripting, Docker / Compose
- Click or Typer for CLI
- React / Next.js (if building dashboards)
- OS: Ubuntu
- LLM tooling: llama.cpp, vLLM, Ollama, GGUF, HuggingFace
- Remote: Tailscale, SSH, or light VPN-based access
ð Why Join ModelVault?
- Help define a new category: secure, local AI for everyone â starting with businesses, expanding to the world
- Work on real, shipped hardware + software that users will touch
- Be part of the founding engineering team with high autonomy and deep ownership
- Remote-first, async-friendly culture with deep focus and high velocity
Please mention the word **COMPREHENSIVE** and tag RMzguNjguMTM0LjE5NA== when applying to show you read the job post completely (#RMzguNjguMTM0LjE5NA==). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.
Apply Now