Software Tips for ML Researchers

https://www.eugenevinitsky.com/posts/quick-software-tips/

• Use Hydra (or Pyrallis) for configs. Don't just write the hyperparameters/config parameters in code files.
• Use package manager to manage python version and packages. E.g. Conda, uv-pip, Poetry, pixi
• Use git, don't use jupyter notebooks (because they don't play nice with git)
• Don't tune hyperparameters by hand. Use Optuna, Ray Tune or custom tuner.
• Run experiments in cluster not in desktop (quick experiments = quicker progress)