Skip to main content
PilotGet a free AI cost & reliability audit: every fix priced in dollars per month. You keep the savings; I get a case study, a testimonial and your feedback.2 of 3 places leftHow the pilot works
Ankit Jha
Portrait of Ankit Jha

Ankit Jha

AI EngineerProduction LLM systems, evals & cost

Open to forward-deployed and applied AI engineering roles

Contract or full-time. I work embedded with teams on production LLM systems: scoping fuzzy problems, building the evals, shipping the fix, and staying accountable for the numbers after launch.

How I work

  • Scope the fuzzy problem with the people who use it.

    Discovery with users and stakeholders first, so the work targets the outcome that matters, not the first symptom.

  • Build the evals before changing anything.

    A golden eval set from production traces, so every later change can be shown to hold quality.

  • Ship the fix in the team's codebase.

    Through their review process and CI, gated by evals, with a rollback plan for each change.

  • Stay accountable for the numbers after launch.

    Cost per successful outcome, eval pass rate and latency, measured against the baseline.

Stack

  • LLMs: Amazon Bedrock, and direct APIs such as OpenAI and Anthropic
  • Evals: golden eval sets, eval gates in CI, LLM judges calibrated against human labels
  • RAG, agents, model routing and caching
  • AWS, backend systems, CI/CD and workflow automation
  • LLM observability and cost: cost per successful outcome, showback, FinOps

Systems I've built

Short write-ups (problem, architecture, evals and outcome) are on the Work page. See the systems

Time zone

  • India (IST), working remotely
  • Overlap: early morning US East, working hours EU, UAE and Singapore, afternoon ANZ

CV

Email me for a CV. A line or two on the role and the team helps me reply with what's relevant.

[email protected]

Prefer a call? Choose "Contract or full-time AI engineering role" in the booking form: Get a free cost check

AI Engineering Economics

Articles on building LLM systems that hold up in production, and what they cost. Each one by email as it goes live; nothing else.

Articles are published on Hashnode.

Get new articles by email