Systems research CUDA · inference · measurement
Ibteshamul
Haque
CUDA kernels and LLM inference systems — and the measurement discipline to tell when something actually got faster.
Horizontal range-bar chart of custom CUDA kernel speedup against a baseline. 3 comparisons; each bar spans the measured minimum to maximum multiplier across independent sessions, so bar width is run-to-run spread, not error. A reference rule marks 1×, meaning no speedup. Each comparison and its exact range follow.
- vs framework baseline
- vs vendor library
- 1× baseline
speedup — × versus baseline · linear scale · higher is faster
- Ranges span independent sessions. The spread is run-to-run variation, not error bars.
GitHub contribution calendar for titoatwork. 837 contributions in the 365 days from 7 Sept 2025 to 6 Sept 2026, on 72 days. The busiest day was 26 Jul 2026 with 98. Counts come from the public profile, so private contributions appear only if that account exposes them. This is a record of activity on one host, not a measure of work done.
- Contributions
- 837
- last 365 days
- Active days
- 72 / 365
- days with at least one
- Busiest day
- 98
- 26 Jul 2026
Counts what the public profile exposes. Private contributions appear only if that account chooses to show them.
No public pushes in the window GitHub reports. The feed is capped at 90 days and 300 events, so quiet periods and private work both read as nothing here.
Currently
Finishing COLIDE — custom CUDA kernels for an IoT intrusion-detection model, now in manuscript prep — and building Slipstream, an LLM serving engine organised around a memory-aware scheduler.
Focus
- 01 Custom CUDA kernels for small, control-flow-heavy models
- 02 LLM inference systems — KV-cache scheduling, quantized decode
- 03 On-device and air-gapped deployment
- 04 Benchmark methodology: ranges, not hero numbers
01 Work
Selected work
4 projects 2 with published figures
COLIDE
Custom CUDA kernels for a CNN-BiLSTM intrusion-detection model, with an on-device LLM that explains alerts without ever sitting in the detection path. A systems study — and an exercise in not overclaiming.- BoT-IoT macro-F1
- 0.9780 ± 0.0033
Slipstream
An LLM inference engine built from scratch, where the research sits in the scheduler — an output-length-predictive policy that decides which sequences get to occupy a full KV cache. No published figures yet 2026 active 03qgemm-mx
Recovering block-scaled FP4 throughput on GPUs that have no native FP4 multiply. The memory saving already ships; the speed is being left on the floor, and this measures exactly where it goes. No published figures yet 2026 active 04RISC-V parameter extraction
Auditing whether a published extraction benchmark measures what everyone reads it as measuring. It did not — the correction was filed upstream, and the fixes it produced were merged.- Merged upstream PRs
- 16
02 Method
Measurement discipline
Re-running the same benchmark suite twice in one sitting moved framework latencies by 14–17%, and moved individual kernel configurations by up to 27% — while each session's own coefficient of variation looked perfectly tight. Within-session variance understates the real uncertainty on a WSL2 development box. Every latency figure here is therefore a range across independent sessions, and single-run tables are treated as unpublishable.
Point estimate with an error bar. BoT-IoT macro-F1 is 0.9780, plus or minus 0.0033, n = 5 (seeds 42–46). The interval therefore runs from 0.9747 to 0.9813. The horizontal axis is truncated: it runs from 0.96 to 0.99, not from zero. Comparators drawn for reference: protocol LightGBM · 0.9818 at 0.9818. No distribution curve is drawn, because the underlying samples are not published.
n = 5 (seeds 42–46)
BoT-IoT macro-F1 · linear scale · truncated axis — window 0.96 – 0.99, not zero-based
full 0–1 scale · the box marks the window plotted above
- Interval spans 0.9747 to 0.9813: the published ± figure, plotted as published. n = 5 (seeds 42–46). The source does not state whether it is a standard deviation, a standard error or a confidence interval, so it is not labelled as one here.
- No distribution is drawn. There is a mean and a spread, not a set of samples — a curve over these two numbers would be a shape nobody measured.
What is not claimed
This section is load-bearing, not modesty.
-
There is no full-pipeline speedup number.
"Custom CUDA is N× faster than TensorRT" would be an easy sentence to write and it would be false.
-
A negative result stands unretracted.
Rather than quietly dropping the comparison, the active claim set treats server Block 3 as unresolved until a fresh run exists.
-
An earlier ToN-IoT result was withdrawn.
The corrected leakage-safe rerun is weaker and honest — CNN 0.8075 against a same-split random forest at 0.9626 — and the old figures are tombstoned in the repository rather than deleted.
Quoted verbatim COLIDE — what is not claimed
03 Writing
Recent notes
Edge LLMs that never block the detection path
Generation takes about eight and a half seconds. Dispatch takes 16.6 microseconds. Which of those two numbers you report decides whether the design is honest.
When frameworks lose on tiny models
TensorRT FP16 was slower than eager PyTorch. ONNX Runtime on the CPU beat several GPU paths. The model was small enough that none of the usual assumptions applied.
On measuring CUDA kernels without lying to yourself
A kernel I had explained away as floating-point noise was a data race. The explanation was wrong in a way I could have checked in one command.
04 Contact