File size: 625 Bytes
982eea0 c6649e6 982eea0 c6649e6 982eea0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | <!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width" />
<title>Distributed LLM Inference Platform</title>
<link rel="stylesheet" href="style.css" />
</head>
<body>
<div class="card">
<h1>Distributed LLM Inference Platform</h1>
<p>Multi-GPU LLM serving: one model worker per GPU, with a load balancer that routes each request to the least-busy worker.</p>
<p>Work in progress. Throughput and latency benchmarks will be published here.</p>
<p>Code: <code>server/worker.py</code> · <code>server/load_balancer.py</code></p>
</div>
</body>
</html>
|