Weekly Updates for Arjun Dahal

Hi Group,

Milestone: Dec 20 ( Paper’s first Draft )

I have created on end-to-end implementation of the approach for one dataset ( Adult ) and obtained the results.

I will be getting results for other two datasets ( COMPAS, CREDIT ) by late afternoon to evening.

Summary of Results:

The t_way_samples are obtained from the latent space representation of the training instances.

The ATN was found to be higher when means of bin was used to generate t_way_samples. ( For t = 4, ATN: 0.775 )

Furthermore, using bin_means from discretized columns resulted in better results for the ratio of discriminatory instances compared to ratio of discriminatory instances from the “test set” of the adult income dataset.

I would like to signup for discussion for feedback on the results & further experiments.

sounds like good progress. also remember to discuss the issue of ground truth/discretization you mentioned during the meeting. please try to have an agenda and also what you expect to get out of the discussion, in terms of action items and decisions to make.

Hi All,

Milestone: Dec 24, Paper’s Internal Deadline

I have completed the following action items:

1. Discriminatory Instances are generated when a random value is selected from the bin obtained using entropy-based discretization.

Using the mean value of the bin results in a slightly better ATN.

1. Comparison with Ankita’s Approach. The current approach has better results than Ankita’s Approach.

I will present briefly the results if the time permits.

I have an implementation of Conditional VAE using Pytorch. However, I am trying to integrate the implementation with the existing library for better results.

I will complete this last action item and write the approach section as soon as possible for the design decision.

great job. now it is time to start on paper writing

Hello Everyone,

Milestone: Dec 24, Paper’s Internal Deadline

I am following up on the action items from the past week:

1. Discriminatory instances generation when a random value is selected from the bin obtained using entropy-based discretization.

1. Comparison with Ankita’s Approach.

1. Implementation of Conditional VAE.

The target date for the action items listed above was today to keep in track with the milestone; however, I was pretty busy this weekend with TA duties. I will follow up on these action items and their results via Slack, by tomorrow evening.

I would suggest you take off “possibly”. mindset matters. things happen only with commitment.

Hello Everyone,

Milestone: Nov 29: Complete Pipeline for current approach along with results

Action Items performed this week:

1. Sampling from the multi-variate normal distribution, t-way coverage selection mechanism to get the data points.

1. Compare ATN ( Ankita’s approach ) to approach (1).

Project Page: https://ajdahal.github.io/portfolio/

I will be presenting on new idea session today.

the project page does not seem to be in a good state. also please summarize the results of your action items, which would be more informative.

Hello Group,

Action Item completed this week:

I have been doing some experiments with regards to VAE for generating data points.

Fixed the issue with kl divergence for ‘hours-per-week’ observed in the last meeting.

I have obtained metrics for Reconstruction Loss vs Number of Embedding Dimension.

ATN and Kldivergence for random binning of embedding dimension.

Currently, I am carrying out t-way combination of latent space vectors. I believe I will have results by this afternoon.

Action Items for next week:

1. Complete the implementation of Conditional VAE, thus completing the pipeline.

1. Review all the Fairness definitions, and present them.

1. I have a list of papers related to my current project regarding VAE and it’s use in fairness testing. I will add state of the art papers regarding fairness testing to the list and upload them to the project page.

I would like to book 15 minutes of time for discussion on the results and the approach.

Hello Group,

I am still working on the experiments regarding fairness testing for Variational AutoEncoder.

I will share some results on slack before Nov 12 regarding the process with some metrics.

since you are targeting IWCT, whose deadline is fast approaching, it would help you can work out a schedule, with break-down tasks and target dates. the schedule will guide all your efforts toward the paper submission.

Hello Everyone,

Milestone: Baseline for Fairness Testing ( Nov 12 )

I am working on making changes on the existing library of VAE to encorporate the combinatorial sampling in it’s latent space. Furthermore, I am also looking into creating conditional VAE and recording metrics for ATN and KL divergence along the way. I am currently working on the implementation.

I will briefly discuss the findings this Friday.

Hi Group,

Milestone: Baseline for Fairness Testing

I have collected and reviewed the necessary literature for the fairness testing with variational auto-encoder and implemented the necessary code for conditional variational auto-encoder. I would like to signup for a brief discussion tomorrow regarding the review and the results.

do you want to put a target date on your milestone?

Hi Group,

Milestone: Finalize the baseline approach Oct 18

I have been working on the following action items since last week:

1. Literature review related to fairness testing using data samples that reflect realistic changes when the sensitive attribute is flipped for fairness testing.

1. Conditional VAE for data generation.

I have made some progress with the action items above, but haven’t worked till completion primarily due to travel.

I will update my progress everyday till coming Friday to ensure the baseline approach is finalized.

try your best to make the milestone

Milestone: Finalize the baseline approach for Fairness testing ( 18 / 08 )

Intermediate Goals:

A. Complete literature search for fairness testing using data samples that reflect realistic changes when sensitive attribute is flipped for fairness testing. ( 11 / 08 )

B. Generation of Data samples conditioned on specific sensitive attribute. ( 11 / 08 )

Action Items completed:

1. Compiled a list of papers that lists papers that use counterfactuals for fairness testing . Adding the description to indicate if the paper is relevant to my task.

1. I have basic code template for conditional VAE. I am debugging it for generating better samples.

I will present today in the topic of “LLM and it’s application in Tabular Data”.

sounds good. i am very interested in hearing what you found out from your literature review first, in your presentation today. also what are the specific issues with conditional VAE?

Thank You Dr. Lei for your feedback. I will be able to complete the literature review by Friday. Furthermore, I will sample few datapoints from the CTVAE and present some results this Friday.

To ensure, I make incremental progress to the task, I will update the status on channel related to my research Wednesday evening.

Hi Group,

I have been following up on action items discussed in the last meeting. I will share some findings regarding LLM producing samples of tabular data for different prompts given to it which will take around ~15 mins. I plan to continue on the action items (sampling in latent space of VAE, generating adversarial examples on top of it ) and present the results in upcoming meetings.

My next milestone would be: Sample instances in latent space of VAE, generate adversarial examples on those samples to produce discriminatory samples and analyze results in each step (Oct 4 ).

this is not a good status updates, as I cannot get useful information out of it.

Hi Group,

I have been looking into the literature regarding Fairness and Clustering since the last meeting

( shared in the channel arjun-research ). Creating Baseline approach with the findings from last few weeks is taking little more time than expected.

I intend to present the summary of findings, approach and summary of relevant literature in the upcoming meeting.

Hello All,

Milestone: Present on literature of all existing work on Fairness testing with clustering ( sep 24 )

Action items for the week: I have been following up on action items ( Divide the data into further sub-clusters to get discriminatory and non-discriminatory clusters, Inspect the datapoints that were discriminatory and non-discriminatory in labels as well as prediction ) discussed in last meeting but haven’t been able to make much progress as I amn’t very well. I will present some findings in upcoming meeting on Friday.

it is probably time to look back at what you have found out, and then start on developing your own approach. sign up a technical discussion to summarize your findings and propose some possible ideas for your own approach.

Hi All,

Milestone:

Improve upon the baseline approach

• Review all the papers regarding Fairness Testing, Active Learning

I have been investigating the clusters that have all the discriminatory samples.

I have some findings that I would like to share in tomorrow’s meeting ( ~15-20 mins ).

try to make a presentation on the major approaches to fairness testing, especially on any existing work based on clustering