Reinforcement Learning
Learning decision policies from reward — I implemented PPO from scratch in PyTorch against custom simulation environments.
Reinforcement learning (RL) trains a policy to act in an environment by maximizing reward over time, rather than learning from labelled examples.
How I use it
In my Independent AI & Full-Stack Engineer (Freelance) work on Real-time Data Systems, I implemented Proximal Policy Optimization (PPO) directly in PyTorch — no RL framework — against a purpose-built simulation environment. Every run was compared with trivial baseline policies run through the same simulator, so a result could show whether the model learned the real dynamics or merely exploited the simulator.
When per-step rewards proved too noisy for PPO to assign credit, I switched the same policy to Evolution Strategies, which optimize the episode outcome directly.
Why it matters
RL is unforgiving: a policy happily learns any flaw in its environment. Building the simulator, the baselines and the evaluation as carefully as the learner is most of the work — the same evaluation-first habit behind Evals for agents.
Related
- Ale Zanello · Evolution Strategies · PyTorch · Deep Learning
- Generative AI Advanced Fine-Tuning for LLMs — covers RLHF-style preference optimization (DPO) from the LLM side