HACKER Q&A
📣 jdthedisciple

How to offer AI services with my H100?


I have a Linux server with Gigabit internet connectivity, which I recently upgraded with a single H100.

Given that models are becoming smaller, more efficient, and faster to run, I am considering to offer AI services (LLMs, RAG, summarization, custom ...) to local clients (professionals, small businesses, private people), who may opt for such services in return for privacy and first level support.

How would I go about setting it up, in terms of

    - resource management
    - resource sharing / load balancing
    - monitoring
    - usage-based charging
Some direction and advice would be greatly appreciated.

Thanks and Happy New Year!


  👤 hekike Accepted Answer ✓
You can check https://openmeter.io for usage metering and billing.

👤 talldayo
If you didn't consider these things before buying an H100 then I frankly wonder why you even bought it in the first place. Was CUDA burning a $30,000 hole in your pocket or something?