Glossary

Model Distillation

Model distillation is a machine learning technique where a compact neural network (student) is trained to replicate the behavior and performance of a larger, more complex network (teacher), enabling high-accuracy predictions on resource-constrained devices while reducing computational overhead by 10-100x.[3]

Model distillation is a machine learning technique where a compact neural network (student) is trained to replicate the behavior and performance of a larger, more complex network (teacher), enabling high-accuracy predictions on resource-constrained devices while reducing computational overhead by 10-100x.[3]