Akhil Shekkari

Designed and implemented a modular AI agent framework in Python, without relying on LangChain, CrewAI, or any existing agent library. Features a think-act reasoning loop that enables LLMs to autonomously chain tool calls across multiple steps to solve complex tasks.

• Agent core: Async-first agent loop with Pydantic data models, structured output support, and execution tracing for full observability
• Tool system + MCP: Extensible @tool decorator with auto function-to-JSON-schema conversion, MCP client for dynamically loading external tool servers, and human-in-the-loop confirmation for dangerous operations
• Memory & context: Hierarchical optimization combining sliding window truncation, tool result compaction, tiktoken counting, and LLM-based summarization to stay within context limits
• RAG pipeline: Text chunking, OpenAI embeddings, and cosine-similarity search to compress long search results before feeding them back to the agent
• Multi-format files: Unified tool for text, CSV, Excel, PDF, images, and audio with multimodal LLM vision/audio analysis

Commercial inference engines like vLLM and TGI are powerful but opaque. Built a full inference engine from scratch to understand every layer of the serving stack — from kernel-level attention to request scheduling.

• FlashAttention: Triton-fused attention kernels with paged KV-cache, achieving 2.3x speedup over Python-level implementations on 4K–16K token workloads
• Speculative decoding: Qwen3-0.6B draft model with Qwen3-4B as verifier, reducing autoregressive decoding latency by ~28% and increasing throughput by ~1.4x
• Dynamic batching: Request scheduler for concurrent inference, improving GPU utilization by 22% and sustaining 1.6x higher tokens/sec vs sequential decoding
• GPU profiling: Nsight Compute to identify memory-bound bottlenecks; optimized tiling and memory access to reach ~78% of peak HBM bandwidth on A100