Blog

The uncomfortable truth about wealth, revenge, and why dark fuel burns just as bright.

How to negotiate from strength - knowing when to speak, when to stay silent, and how to leave room for both sides to win.

A letter from my future self on becoming undefinable, building a personal monopoly, and why the paradox is the edge.

RL foundations for LLMs: policy gradients, baselines for variance reduction, GRPO implementation details, and practical training considerations for reasoning models.

Advanced RL for alignment: PPO implementation details, GRPO as a simpler alternative, overoptimization risks, and case studies from DeepSeek R1, Kimi K1.5, and Qwen 3.

Post-training for helpful assistants: supervised fine-tuning on instructions, safety tuning, RLHF with preference data, PPO vs DPO, and the challenges of learning from human feedback.

Data filtering and deduplication at scale: n-gram language models, fastText classifiers, importance sampling, MinHash, LSH, and Bloom filters for efficient web-scale processing.

Training data for LLMs: Common Crawl processing, quality filtering, the evolution of data pipelines from BERT to modern models, and the critical role of copyright and licensing.

LLM evaluation beyond accuracy: perplexity, knowledge benchmarks, instruction-following, agent tasks, safety, and why evaluation design shapes what models become.

Practical scaling: muP for hyperparameter transfer, WSD learning rate schedules, case studies from Cerebras-GPT, MiniCPM, and DeepSeek on compute-optimal training.

LLM inference optimization: understanding the prefill vs decode split, KV cache management, speculative decoding, and why inference is fundamentally memory-bound.

I went back to where it started. This time with Shreyasi, clear winter skies, and lungs that didn't burn. No snow, but clear views. The mountains were the same. I was fitter.

Understanding scaling laws: how loss depends on data, parameters, and compute, the Chinchilla tradeoff for compute-optimal training, and why power laws emerge in deep learning.

Hands-on distributed training: implementing collectives with PyTorch and NCCL, data/tensor/pipeline parallelism in practice, and understanding the compute-memory-communication tradeoff.

Distributed training fundamentals: data parallelism, ZeRO/FSDP for memory efficiency, tensor and pipeline parallelism, and how to combine strategies for frontier-scale models.

Writing efficient GPU kernels with Triton: profiling, benchmarking, kernel fusion, and when to hand-optimize versus using torch.compile.

GPU fundamentals for LLM training: memory hierarchy, arithmetic intensity, kernel optimization, FlashAttention, and bandwidth limits.

Mixture of Experts (MoE): adding capacity without proportional compute, routing, load balancing, and what makes MoE stable.

What modern LLMs converge on: pre-norm, RMSNorm, SwiGLU, RoPE, and stability tricks.

Resource accounting for LLM training: compute estimates, memory budgets, dtypes, tensors, and mixed precision.

A tour of modern LLMs, the "bitter lesson" through the lens of efficiency, and why BPE tokenization matters.

My notes on Stanford CS336: Language Modeling From Scratch: how to build a large language model end to end, from data to deployment.

My first Himalayan trek. Unprepared. Unfit. Soaked by monsoon storms. I finished it anyway, and came back wanting more.

The cost of neglecting your mind and the power of intentional attention.

A messy mind is usually a systems problem. Fix it through diet, sleep, environment, and disciplined iteration.

Notes from Herb Cohen's classic on negotiation. Everything is negotiable. You have more power than you think. Learn to see it and use it.

Notes from Ichiro Kishimi and Fumitake Koga's dialogue on Adlerian psychology. Freedom is being disliked by others. It proves you are living by your principles.

Notes from Maxwell Maltz's classic on self-image psychology. Your results aren't limited by talent. They're limited by what you accept as true about yourself.

Notes from Steven Pressfield's manifesto on creative resistance. The hard part isn't the work itself. The hard part is sitting down.

Notes from Michael Gerber's classic on why most small businesses fail. If your business depends on you, you don't own a business. You have a job.

The hidden cost of unearned gifts and the dignity found in earning your way.

Cooperation, trust, and building with others is the only way to scale beyond individual effort.

Reading people, protecting trust, and staying sharp without becoming bitter.

Facing competition, choosing battles wisely, and winning the fights that decide who you become.

The race from chatbots to agents is on. Notes and lessons from Y Combinator's AI Camp on building in the agent era, from Sam Altman to Andrej Karpathy.

Building options, avoiding single points of failure, and staying free by keeping multiple paths to the same destination.

Idiosyncratic means different. If you want mastery, you have to think differently. Normal gets normal. Here's how to build your own worldview.

Time is the one variable nobody accounts for, and the one variable that has the biggest effect on everything. Here's how to use it as a friend, not an enemy.

Decision-making is the one skill no one teaches you. Here's how to master it, step by step, in plain language.

A 10-day journey through rhododendron forests, suspension bridges, and high-altitude meadows to stand face-to-face with Kanchenjunga.

Building character, skill, and standards matters more than chasing money. Diligence is the real source of wealth.

Success is the sum of your human will. Your will to prepare. Your will to sacrifice. Your will to stay consistent when it's boring, hard, and lonely.

Luck is not something you wait for. It's something you design through decisions, standards, and time.

Why luck favors those who move. Courage is not a personality. It's a decision you make in a moment, then again the next day.

The cost of excuses and the power of holding a higher standard.

Taking full responsibility for your life, building discipline, and earning your own path.

The power of action over endless planning. Nothing changes until you move.

Modern life isn't just distracting. It's built to steal your focus. Take it back.