Asynchronous Error Handling Is Hard
11 by hedgehog | 1 comments on Hacker News.
Monday, 30 June 2025
Sunday, 29 June 2025
New top story on Hacker News: Show HN: A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)
Show HN: A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)
3 by mrqjr | 1 comments on Hacker News.
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://ift.tt/Zyr32Rs The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services. It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via Anthropic) • Local endpoints (custom/self-hosted) You can also self-host it with docker-compose. Config is clean, adding a new provider only requires a simple plugin-style addition. Would love feedback, PRs, or even test reports from APIs you’re using. Especially interested in how some lesser-known services compare.
3 by mrqjr | 1 comments on Hacker News.
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://ift.tt/Zyr32Rs The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services. It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via Anthropic) • Local endpoints (custom/self-hosted) You can also self-host it with docker-compose. Config is clean, adding a new provider only requires a simple plugin-style addition. Would love feedback, PRs, or even test reports from APIs you’re using. Especially interested in how some lesser-known services compare.
Saturday, 28 June 2025
Friday, 27 June 2025
Robert F. Kennedy Jr. Tells Fox News The Rotten Truth About His Anti-Fluoride Crusade
from Yahoo News - Latest News & Headlines https://ift.tt/EBTqe4L
Thursday, 26 June 2025
Jill Biden's 'work husband' runs for cover as privilege protection crumbles
from Yahoo News - Latest News & Headlines https://ift.tt/zIqGN3s
Wednesday, 25 June 2025
Tuesday, 24 June 2025
Monday, 23 June 2025
Fart walking — do this 10-minute indoor walking workout immediately after eating to lower your blood sugar, aid weight loss
from Yahoo News - Latest News & Headlines https://ift.tt/ThlSu1B
Subscribe to:
Posts (Atom)