Arjun:
Hi Group,
This week, I have been working on the implementation of my current work on Fairness Testing. I have some initial results however, there are some issues I am facing. Once I have more results, I will update my Slack channel with them.
I have a new idea presentation tomorrow, where I will be presenting this work:
Chen, Canyu, et al. “When fairness meets privacy: Fair classification with semi-private sensitive attributes.” arXiv preprint arXiv:2207.08336 (2022).
Arjun:
Hello Everyone,
Milestone: Complete the current project and have a draft ( ~ Apr 2 )
I was traveling during the spring break.
After my return, I worked on the existing implementation. I have nearly completed the implementation for the current project.
I will be sharing the results of end-to-end implementation in the meeting or by the end of the day.
I will be updating the status of the progress every day at the end of the day till the completion of the project.
Replies:
Dr. Lei:
make a commitment by first removing ~ in the target date
Arjun:
Hello Everyone,
This week I created the artifacts for IWCT 2025. I also carried out the literature review for the paper I am currently working on.
Currently, I am still focused on implementation of the current project I am working on.
I am lagging behind in the project, I will update status of daily progress everyday till the completion of the first draft in my channel.
Replies:
Dr. Lei:
try to be iterative. try to have a baseline version first and then improve upon it.
Arjun:
Hello Everyone,
This week, I focused on the problem of changing the sensitive attribute of tabular datasets, for which I studied a couple of possible techniques ( along with their pros and cons ) and presented them. Furthermore, I worked on the comments from the previous paper IWCT and made the source code ready to work in a virtual environment along with the necessary document that instructs how to run the code. I have revisited the implementation of the Fader networks.
I am looking into the papers that I could compare our work with regarding counterfactual fairness.
Replies:
Dr. Lei:
sounds good. i just sent you some comments on the iwct paper from nist.
Arjun:
Hello Group,
Milestone: Complete the undergoing project ( ~ Mar 4 ).
Action Item Completed:
I will present some results today regarding the flipping of the sensitive attribute and take action based on today’s discussion.
Replies:
Dr. Lei:
sounds good. i suggest you target ICSE which has the submission deadline in about one week
Arjun:
Hello Everyone,
This week, I have been reading some literature for my use case, flipping sensitive attributes for fairness testing respecting domain constraints.
I found this work to be closely related to my work:
Lample, Guillaume, et al. “Fader networks: Manipulating images by sliding attributes.” Advances in neural information processing systems 30 (2017). I am looking forward to a brief discussion tomorrow.
Replies:
Dr. Lei:
try to make your report more informative. what are the major findings, and what you consider this paper to be related.
Arjun:
Hello Everyone,
Milestone: Feb 7 ( Baseline ): Includes end-to-end implementation of two types of VAE models
Action Items completed from the past week:
Conditional VAE was used similarly to how a conditional VAE is used for image data, and the ATN found was ~0.75 for the adult income dataset.
The current implementation needs to be improved for end-to-end results.
I will present tomorrow for the New Idea session.
Replies:
Dr. Lei:
what do you mean by end-to-end implementations? does it mean end-to-end for fairness testing? in the following days, i suggest you focus on the topic of how to flip sensitive attributes. first try to find all the existing work on this problem or related to this problem. second try to develop an algorithm to do the flipping. ask the question, what type of fairness would your algorithm/end-to-end approach test? third try to have an initial implementation to validate the algorithm.
Arjun:
Hello Everyone,
Milestone: Feb 7 ( Baseline )
| Implement a basic version of the project end-to-end for one or more datasets ( DisentangledVAE - ConditionalVAE ) and get the results | Initial Verification |
The end-to-end implementation is carried out to understand the project better and find the potential issues during each step.
Action Items completed this week:
Action Items for next week:
Replies:
Dr. Lei:
good results on the use of disentangled VAE. i would suggest an incremental approach. now you have some initial confidence on disentangled VAE. try to move onto conditional VAE on how to flip the sensitive attributes. try to think deep, in terms of what are the fundamental issues in this problem, what are the possible approaches to address the problem, what approaches you want to take and why, in terms of its pros/cons.
Arjun:
Hello Everyone,
I had a paper submission on Jan 10. I worked on the paper submission before the holidays.
For the past week, I revisited the paper for the necessary fixes mentioned by Dr. Lei. Furthermore, I reflected on this project I had undertaken since Spring 2024. I have a few slides to share regarding this project and paper submission. I want to book 30 mins of the time for the presentation.
New Action Items: I am looking forward to a new project regarding using Disentangled VAE and Conditional VAE for fairness testing.
Replies:
Dr. Lei:
congrats on your first submission. the two directions are indeed good ones to explore. i suggest you focus on the use of Disentangled VAE in the following week, since this is relatively clear in terms of what to do. that is, try to do some experiments to see if its use helps improve the naturalness.
Arjun:
Hello Everyone,
| Milestone: Dec 20 | Internal Deadline for the paper Draft |
What is Completed?
Pipeline for the End-to-End Implementation.
What is pending?
Design Decisions, primarily use of the columns and how to flip the sensitive attribute.
Writeup for the Experiment, Results, and sections.
I propose two forms of processing for the dataset.
All the columns included in AI360 are used for the datasets (Adult, Compas, Credit)
I would like to sign up for a discussion tomorrow.
Replies:
Dr. Lei:
do you have a first draft of the three sections, Background, Approach, and Experiments?
Arjun:
Hi Group,
Milestone: Dec 20 ( Paper’s first Draft )
I have created on end-to-end implementation of the approach for one dataset ( Adult ) and obtained the results.
I will be getting results for other two datasets ( COMPAS, CREDIT ) by late afternoon to evening.
Summary of Results:
The t_way_samples are obtained from the latent space representation of the training instances.
The ATN was found to be higher when means of bin was used to generate t_way_samples. ( For t = 4, ATN: 0.775 )
Furthermore, using bin_means from discretized columns resulted in better results for the ratio of discriminatory instances compared to ratio of discriminatory instances from the “test set” of the adult income dataset.
I would like to signup for discussion for feedback on the results & further experiments.
Replies:
Dr. Lei:
sounds like good progress. also remember to discuss the issue of ground truth/discretization you mentioned during the meeting. please try to have an agenda and also what you expect to get out of the discussion, in terms of action items and decisions to make.
Arjun:
Hi All,
Milestone: Dec 24, Paper’s Internal Deadline
I have completed the following action items:
Using the mean value of the bin results in a slightly better ATN.
I will present briefly the results if the time permits.
I have an implementation of Conditional VAE using Pytorch. However, I am trying to integrate the implementation with the existing library for better results.
I will complete this last action item and write the approach section as soon as possible for the design decision.
Replies:
Dr. Lei:
great job. now it is time to start on paper writing
Arjun:
Hello Everyone,
Milestone: Dec 24, Paper’s Internal Deadline
I am following up on the action items from the past week:
The target date for the action items listed above was today to keep in track with the milestone; however, I was pretty busy this weekend with TA duties. I will follow up on these action items and their results via Slack, by tomorrow evening.
Replies:
Dr. Lei:
I would suggest you take off “possibly”. mindset matters. things happen only with commitment.
Arjun:
Hello Everyone,
Milestone: Nov 29: Complete Pipeline for current approach along with results
Action Items performed this week:
Project Page: https://ajdahal.github.io/portfolio/
I will be presenting on new idea session today.
Replies:
Dr. Lei:
the project page does not seem to be in a good state. also please summarize the results of your action items, which would be more informative.
Arjun:
Hello Group,
| Milestone: Baseline for Fairness Testing | Nov 12 |
Action Item completed this week:
I have been doing some experiments with regards to VAE for generating data points.
Fixed the issue with kl divergence for ‘hours-per-week’ observed in the last meeting.
I have obtained metrics for Reconstruction Loss vs Number of Embedding Dimension.
ATN and Kldivergence for random binning of embedding dimension.
Currently, I am carrying out t-way combination of latent space vectors. I believe I will have results by this afternoon.
Action Items for next week:
I would like to book 15 minutes of time for discussion on the results and the approach.
Arjun:
Hello Group,
| Milestone: Nov 12 | Baseline for Fairness Testing |
I am still working on the experiments regarding fairness testing for Variational AutoEncoder.
I will share some results on slack before Nov 12 regarding the process with some metrics.
Replies:
Dr. Lei:
since you are targeting IWCT, whose deadline is fast approaching, it would help you can work out a schedule, with break-down tasks and target dates. the schedule will guide all your efforts toward the paper submission.
Arjun:
Hello Everyone,
Milestone: Baseline for Fairness Testing ( Nov 12 )
I am working on making changes on the existing library of VAE to encorporate the combinatorial sampling in it’s latent space. Furthermore, I am also looking into creating conditional VAE and recording metrics for ATN and KL divergence along the way. I am currently working on the implementation.
I will briefly discuss the findings this Friday.
Arjun:
Hi Group,
Milestone: Baseline for Fairness Testing
I have collected and reviewed the necessary literature for the fairness testing with variational auto-encoder and implemented the necessary code for conditional variational auto-encoder. I would like to signup for a brief discussion tomorrow regarding the review and the results.
Replies:
Dr. Lei:
do you want to put a target date on your milestone?
Arjun:
Hi Group,
Milestone: Finalize the baseline approach Oct 18
I have been working on the following action items since last week:
I have made some progress with the action items above, but haven’t worked till completion primarily due to travel.
I will update my progress everyday till coming Friday to ensure the baseline approach is finalized.
Replies:
Dr. Lei:
try your best to make the milestone
Arjun:
Milestone: Finalize the baseline approach for Fairness testing ( 18 / 08 )
Intermediate Goals:
A. Complete literature search for fairness testing using data samples that reflect realistic changes when sensitive attribute is flipped for fairness testing. ( 11 / 08 )
B. Generation of Data samples conditioned on specific sensitive attribute. ( 11 / 08 )
Action Items completed:
I will present today in the topic of “LLM and it’s application in Tabular Data”.
Replies:
Dr. Lei:
sounds good. i am very interested in hearing what you found out from your literature review first, in your presentation today. also what are the specific issues with conditional VAE?
Arjun:
Thank You Dr. Lei for your feedback. I will be able to complete the literature review by Friday. Furthermore, I will sample few datapoints from the CTVAE and present some results this Friday.
To ensure, I make incremental progress to the task, I will update the status on channel related to my research Wednesday evening.
Arjun:
Hi Group,
I have been following up on action items discussed in the last meeting. I will share some findings regarding LLM producing samples of tabular data for different prompts given to it which will take around ~15 mins. I plan to continue on the action items (sampling in latent space of VAE, generating adversarial examples on top of it ) and present the results in upcoming meetings.
My next milestone would be: Sample instances in latent space of VAE, generate adversarial examples on those samples to produce discriminatory samples and analyze results in each step (Oct 4 ).
Replies:
Dr. Lei:
this is not a good status updates, as I cannot get useful information out of it.
Arjun:
Hi Group,
I have been looking into the literature regarding Fairness and Clustering since the last meeting
( shared in the channel arjun-research ). Creating Baseline approach with the findings from last few weeks is taking little more time than expected.
I intend to present the summary of findings, approach and summary of relevant literature in the upcoming meeting.
Arjun:
Hello All,
Milestone: Present on literature of all existing work on Fairness testing with clustering ( sep 24 )
Action items for the week: I have been following up on action items ( Divide the data into further sub-clusters to get discriminatory and non-discriminatory clusters, Inspect the datapoints that were discriminatory and non-discriminatory in labels as well as prediction ) discussed in last meeting but haven’t been able to make much progress as I amn’t very well. I will present some findings in upcoming meeting on Friday.
Replies:
Dr. Lei:
it is probably time to look back at what you have found out, and then start on developing your own approach. sign up a technical discussion to summarize your findings and propose some possible ideas for your own approach.
Arjun:
Hi All,
Milestone:
Improve upon the baseline approach
I have been investigating the clusters that have all the discriminatory samples.
I have some findings that I would like to share in tomorrow’s meeting ( ~15-20 mins ).
Replies:
Dr. Lei:
try to make a presentation on the major approaches to fairness testing, especially on any existing work based on clustering
© 2023 Jeff Lei's Lab.
Site made with Jekyll. Template from the Allan Lab
We are part of the CSE Department at University of Texas at Arlington.