Day 20: GANs, Diffusion, and Final Project Launch

Agenda

• 3:45-4:05pm: CNN Shareout
• 4:05-4:35pm: GANs and Diffusion Models
• 4:35-5:25pm: Final project launch

CNN Shareout (15 minutes)

At your tables, discuss what you found doing assignment 18. We’ll make a list of key takeaways together as a class.

GANs and Diffusion Models (30 minutes)

GANs and Diffusion Models are methods of generating synthetic images, videos, or audio. These tools have gotten so good, that the lines between real and fake have become very hazy. Some interesting examples can be found in the isthisAI subreddit (for example).

A nice overview of GANS.

There is lots of code out there with varying degrees of complexity, but one thing to note is that we’ve learned many of the basic building blocks. Example: https://github.com/sw-song/PyTorch-GAN/blob/master/implementations/dcgan/dcgan.py

Here is a really great conceptual overview of diffusion models (including guided diffusion where a text prompt guides the image generation process.)

Ethical issues in image generation

Here is a well-cited paper on the effects of these tools on artists . The paper goes through the various cultural and economic harms that these tools bring and discusses relevant copyright statutes.

AI-driven media generation poses all sorts of potential issues for democracy. Here is a law review article from ages ago (2019!) that systematically goes through risks and potentially safe guards.

Bonus: Explore some fun image generation applications

We probably won’t get to these in class, but they could be interesting to explore (and may also help you think about your final project deliverables):

• https://mitmedialab.github.io/GAN-play/
• https://magenta.tensorflow.org/assets/sketch_rnn_demo/index.html
• https://github.com/nashory/gans-awesome-applications
• https://github.com/sw-song/PyTorch-GAN
• https://github.com/ZZUTK/Face-Aging-CAAE

Overview of Final Project Structure

We’ll go through a slide deck to introduce the final project

A more detailed walkthrough of the final project deliverables, due dates, and assessment is also available (note: some timings are still being finalized

Discuss Projects at Tables

You had some time to think of final project topics and why they are important to you. Take some time and share with folks around you (aim for a group size of around 5) what you came up with. As a listener feel free to offer additional ideas, mutual interest in the project, or useful clarifying questions (we won’t have time for long discussions though).

Idea Clustering and Team Formation

Write your favorite ideas on Post-It notes and include your name. Place the Post-Its on the board in the appropriate category. Once the ideas are clustered, you can congregate by one of the clusters to find students who are interested in similar topics. Through this process we hope you will find someone to work with or additional ideas for your own project topic (if you are working alone).

If you are not able to find a partner (and want one) or could work alone but are open to having a partner, please fill out this survey no later than 5pm on Tuesday, April 7th. If you have a partner already, you do not (and should not) fill out this survey.