Prototypical methods have recently gained a lot of attention due to their intrinsic interpretable nature, which is obtained through the prototypes. With growing use cases of model reuse and distillation, there is a need to also study transfer of interpretability from one model to another. We present Proto2Proto, a novel method to transfer interpretability of one prototypical part network to another via knowledge distillation. Our approach aims to add interpretability to the "dark" knowledge transferred from the teacher to the shallower student model. We propose two novel losses: "Global Explanation" loss and "Patch-Prototype Correspondence" loss to facilitate such a transfer. Global Explanation loss forces the student prototypes to be close to teacher prototypes, and Patch-Prototype Correspondence loss enforces the local representations of the student to be similar to that of the teacher. Further, we propose three novel metrics to evaluate the student’s proximity to the teacher as measures of interpretability transfer in our settings. We qualitatively and quantitatively demonstrate the effectiveness of our method on CUB-200-2011 and Stanford Cars datasets. Our experiments show that the proposed method indeed achieves interpretability transfer from teacher to student while simultaneously exhibiting competitive performance.
@inproceedings{Keswani2022Proto2ProtoCY,abbr={CVPR},title={Proto2Proto: Can you recognize the car, the way I do?},bibtex_show={true},author={Keswani, Monish and Ramakrishnan, Sriranjani and Reddy, Nishant and Balasubramanian, Vineeth N.},booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)},eprint={2204.11830},archiveprefix={arXiv},code={https://github.com/archmaester/proto2proto},pdf={https://arxiv.org/pdf/2204.11830.pdf},year={2022},selected=true,website={/projects/proto2proto}}
2021
BMVC
Beyond Classification: Knowledge Distillation using Multi-Object Impressions
Nayak, Gaurav Kumar*,
Keswani, Monish*,
Seshadri, Sharan,
and Chakraborty, Anirban
Knowledge Distillation (KD) utilizes training data as a transfer set to transfer knowledge from a complex network (Teacher) to a smaller network (Student). Several works have recently identified many scenarios where the training data may not be available due to data privacy or sensitivity concerns and have proposed solutions under this restrictive constraint for the classification task. Unlike existing works, we, for the first time, solve a much more challenging problem, i.e., “KD for object detection with zero knowledge about the training data and its statistics”. Our proposed approach prepares pseudo-targets and synthesizes corresponding samples (termed as “Multi-Object Impressions”), using only the pretrained Faster RCNN Teacher network. We use this pseudo-dataset as a transfer set to conduct zero-shot KD for object detection. We demonstrate the efficacy of our proposed method through several ablations and extensive experiments on benchmark datasets like KITTI, Pascal and COCO. Our approach with no training samples, achieves a respectable mAP of 64.2% and 55.5% on the student with same and half capacity while performing distillation from a Resnet-18 Teacher of 73.3% mAP on KITTI.
@article{Nayak2021BeyondCK,abbr={BMVC},title={Beyond Classification: Knowledge Distillation using Multi-Object Impressions},author={Nayak, Gaurav Kumar* and Keswani, Monish* and Seshadri, Sharan and Chakraborty, Anirban},eprint={2110.14215},bibtex_show={true},archiveprefix={arXiv},code={https://github.com/archmaester/moid},pdf={https://arxiv.org/pdf/2110.14215.pdf},year={2021},selected=true,journal={BMVC},volume={abs/2110.14215},html={https://www.bmvc2021-virtualconference.com/conference/papers/paper_0906.html}}