shadcn.io is not affiliated with official shadcn/ui
Inference Latency Monitoring Block for React Aria
Median and p99 latency for three models in three regions as stacked bars of queue, preprocess, model and postprocess on one millisecond axis, a region toggle, a sentence naming the slowest tail, and a table of batch size, throughput and p99.
Installation
Overview
See where the tail of a model's latency comes from, region by region, with this inference latency panel for React with React Aria. It is Servewhistle, a model-serving monitor (made up), timing three models over the last hour. A hairline frame with a plus mark on each corner holds, for each model, two horizontal bars on one millisecond axis, the median above the 99th percentile, each split into queue wait, preprocessing, the model itself and postprocessing. The parts differ by fill and hatching, not colour. A Toggle Group in the header picks the region, and the sentence under the heading names the model with the slowest tail there and says how much of its p99 is queue wait against how much of its p50. On a phone the four parts are listed under each pair of bars. A table under the bars gives each model's batch size, throughput and p99. Guides draw the frame and its bands; the entrance is CSS only. For ML platform teams, serving owners and anyone tuning batch sizes and replicas.
FAQ
How do I use my own timings?
Edit LINES: each line is a region, a model, a batch size, requests a second, then the four parts of the median request and the four parts of the 99th percentile request in milliseconds, for example eu-west|Heron Large|16|52|41,6,214,8|610,15,342,24. Bars, totals, the header sentence, the table and the hidden table for screen readers all derive from those numbers. The axis runs to TOP milliseconds.
Why is queue wait singled out?
Because it is where tails usually come from: the model's own time barely changes between the median and the p99, while the wait for a free replica can grow tenfold under load. The sentence compares queue's share of the p99 with its share of the p50 for the slowest model in the region, so a long queue stands out. The percentile parts are measured separately, so they do not have to add up across p50 and p99.
What does the region toggle change?
Only which three rows are drawn. The axis stays at 0 to 1,000 ms in every region so bars compare across them, the panel keeps its height, and a polite status line reads the new sentence.
What is real here and what is typed?
All of it is typed: the models, regions, timings, batch sizes and throughput are examples, and nothing is served or timed. The product and the models are made up. In your app feed the same shapes from your serving metrics, one cell per model and region.