Weekly Updates for Shovon Niverd Pereira

Shovon:

Milestones:

1. Create a pipeline with ART at the first part and Knowledge distillation at the latter part. (Done).

1. Get the baseline to work properly.(July 16)

1. After the baseline is working, replace the normal distribution with ART.(July 21)

Action Items:

1. Train a student model(CNN) with data generated by a GAN and labels generated from the teacher model(CNN), but the noise vector will be drawn from normal distribution

1. Later, do the same, but this time, the noise vector will be drawn from ART.

1. Compare performance. Tackle any issue arising in between

Results/Findings:

1. During the running experiments with the current baseline faced problems with the codebase.

1. Some libraries were old and had to be replaced

1. The code did not have a teacher model. I tried pretrained models, tried training my own, and finally, training a model with their defined architecture with CIFAR 10 worked.

1. Currently, the accuracy of the student model is at 62%. A few more iterations will increase it.

Questions/issues

1. Spent quite a bit of time getting the code running. Right now, apparently, there are no issues, but I would like to discuss some actions I want to take next

1. Datasets to be used

1. Models to be used

1. The training approach to be taken

Replies:

Dr. Lei:

good that you are making some progress. try to stay focused . the goal at this point is to have some initial feedback about whether it makes sense to replace a general random sampling process with ART in a GAN-based network for synthetic data generation.

---

Shovon:

Milestones:

1. Create a pipeline with ART at the first part and Knowledge distillation at the latter part. (July 10).

Action Items:

1. Explored:

1. Explored the idea of the DFMS-HL paper(https://openaccess.thecvf.com/content/CVPR2022/papers/Sanyal_Towards_Data-Free_Model_Stealing_in_a_Hard_Label_Setting_CVPR_2022_paper.pdf#page=2.88)

1. Ongoing

1. Replacing the pretraining module of the KD paper with ART

Results/Findings:

1. Found a paper that provides code for KD: https://sites.google.com/view/dfms-hl

1. DFMS-HL uses a pretraining stage to prepare a generator to generate more realistic images.

1. They use random image priors to train the generator

1. Found another potential technique, which uses public domain distribution to ensure the naturalness of the synthetic data:https://arxiv.org/pdf/2109.14857

Questions/issues

1. Still not sure how they randomly generate the image priors.

1. Last week did not have much progress. I was stuck understanding the existing codebase.

1. Need a technical discussion at the meeting.

Shovon:

Milestones:

1. Create a pipeline with ART at the first part and Knowledge distillation at the later part. (July 5), need more time due to a knowledge gap.

Action Items:

1. Explored:

1. Explored the idea of the DFMS-HL paper(https://openaccess.thecvf.com/content/CVPR2022/papers/Sanyal_Towards_Data-Free_Model_Stealing_in_a_Hard_Label_Setting_CVPR_2022_paper.pdf#page=2.88)

1. Looked at their codebase to find the random generation module

1. Need to do:

1. Start replacing random generation with ART, before that need to figure out a to fit ART.

Results/Findings:

1. Found a paper that provides code for KD: https://sites.google.com/view/dfms-hl

1. DFMS-HL uses a pretraining stage to prepare a generator to generate more realistic images.

1. They use random image priors to train generator

1. I think if the genrator is pretrained with better diverse data then the whole train process will be faster

Questions/issues

1. Still not sure how they randomly generate the image priors.

1. I am quite confident ART will fit in the pretrain process but need to figure how ART can generate random images.

No update was provided for the week ending 2025-04-30.