Memory-efficient neural network training

This invention describes a method to train large neural networks more efficiently by saving computer memory. It works by discarding temporary calculation data immediately after a network part finishes its initial pass, then re-calculating it on demand for fine-tuning. This approach is especially useful for distributed networks where different parts are processed by separate computers.