Infrastructure for LLM systems
Backend that stays calm
when things break.
I'm Bishal Hona, a backend engineer. I build the parts of AI products that live between the model and the person using it. The background jobs, the memory that has to survive across sessions, the streaming, the auth. The stuff that's invisible when it works.
Plenty of AI products look great in a demo. I build the part that keeps them working once real people are using them.
- 01 · Isolated queues
- I run the background AI jobs in separate queues, so if one kind of task breaks it can't drag the rest down with it. And if a job gets stuck partway, a recovery process picks it back up on its own instead of letting it disappear.
- 02 · Memory that lasts
- I build memory that carries from one session to the next. I keep a tight cap on how much of it goes to the model at once, and I record which context produced a given response so I can go back and check.
- 03 · Streaming responses
- Responses stream in as the model writes them, so nobody sits watching a blank screen while they wait for the whole answer to land.
- 04 · Session security
- Sessions rotate on their own. If a stolen session token ever gets reused, I log every one of that user's sessions out right away.