"Can you keep your Instagram active forever" was the question I've been asking myself before I started this project.

Using Social Media automation tools it's already possible to schedule content years ahead. But how can we keep creating fresh & unique content that long ahead in time?

The answer? A.I.

@notsabatino is my A.I. bot that lives its own life and documents his life on Instagram.

Let's dive into the details!

Creating the deepfake model

Stable Diffusion and Dreambooth have been wildly popular in the second half of 2022. Profile picture generator tools took the internet by storm (I even created my own!) and people could suddenly create deepfake models of themselves for a few bucks.

Using my own tool, I created my deepfake model, but if you're a bit technical, you can create these deepfake models yourself (for free) using a Google Colab (eg. using this repo).

I created a dozen of deepfake models until I was satisfied with the result, at the time of this article (february 2023) these variables worked best:

• 15 close-up photos of my face in various conditions (body cropped off*)
• class name: man
• model: stable diffusion 1.5
• learning rate: 1e-6
• steps: 2000

*Unless you have a very special body type, you can easily write a prompt that will output your body type

(Chat)GPT prompts

Now that we had a deepfake model we could use, the next problem to solve is to actually write the prompts in an automated way.

To start out, I thought it'd be fun to make my clone travel Italy for 3 months. Using ChatGPT I quickly iterated on some prompts, that I could then feed into GPT-3 with an API. I split the prompts up in 2 parts:

• Generate a 3-month itinerary (macro)
• Generate activities in a specific place (micro)

Generating an itinerary

To get a 3-month itinerary, this is the script I used:

ChatGPT would then answer something like this:

The places seem to be jumping all over Italy, but that's not an issue at this point. I plotted these locations on a map and there we have it, our travel itinerary was ready:

Finally, I calculated an optimal route between the locations and we could move on to the next task.

Generating activities

To generate activities per place, I used the same prompt as above, but with a slight difference:

So every day at a random time, this prompt gets executed and GPT will respond something like:

It's likely that at 15:00 on February 8th I might be taking a gondola ride.

Note that the weather variable is real data, fed to the prompt using an API. This makes the mixed-reality feeling even better.

Visualizing an activity

Now that we have our text-based activity, we can make it visual. In my homelab I run my own 4090 GPU, this allows me to iterate prompts at an insane speed. After prompt-hacking for multiple nights in a row, I created a base prompt that is flexible enough to handle almost everything I throw at it:

This would result in this (cherry-picked) image:

If you took a close look at the prompt, you see we're asking Stable Diffusion to output a medium shot. We do this because our deepfake model is trained on my face specifically. Stable Diffusion is very good at recreating my face, so in this part, we put all our efforts into creating a high quality face. We worry about the body later.

Next up, we have to create the rest of the image, since an Instagram with close-up shots of my face will be quite boring.

I experimented a lot with Stable Diffusion inpainting, but the results were very bad. So I decided to move on to Dall-E for this step.

Using Dall-E to outpaint the image

Before we send the image to Dall-E, we generate a 1024x1024 canvas, and paste the initial image in it. We leave the other pixels transparent, and push it to the inpainting API.

I created a repository that demonstrates this technique.

Together with this image, we send a prompt to Dall-E as well. I found that using simple prompts works best (my theory is that they do some magic on it behind the scenes):

Realistic photo of a man in a gondola in venice

And the API will then respond with our full image:

Posting to Instagram

The final step in our process is actually posting the image to Instagram. Using the Content Publishing API we can automatically post our images. This feature is only available on business accounts.

The description of the Instagram post is also generated with GPT, eg:

Cruising through the beautiful city of Venice in style #gondola #venice #italy

Automating everything

Now that we have all individual components, it's time to automate (almost) everything. I set up a simple Laravel application that schedules a daily cronjob at a random time. This cronjob would set a chain of actions in motion, a quick schematic reveals how everything plays together:

@notsabatino runs on an M2 Mac Mini, which runs both the Laravel application as well as the stable diffusion inference server. Every image takes about 20-30 seconds to generate, but that's perfectly fine since this entire bot runs asynchronously.

This setup generates 20 images at a time and then sends out an email so I can cherry pick the images I like best:

I'm confident this step can be entirely automated, but I'm not sure I'm quite ready for that yet 😬

After I 'voted' on my preferred images, the chain continues and a random post makes it to Instagram.

Final words

Creating this bot has been a very fun experiment, but at the same time it made me contemplate the ethical implications of A.I. even further. Preserving an individuals online presence may seem innocent, but in the wrong hands all kinds of fake news can be generated with a simple prompt. That's why (for the time being) I'd like to be 100% in control for cherry-picking the images that make it out to the internet.

Once I'm over that mental hurdle, I might schedule posts days, weeks or even months in advance. Until then, @notsabatino keeps posting once daily at a random time, after I've given my permission 😉

---

Follow me on Twitter: https://twitter.com/SabatinoMasala

Follow @notsabatino on Instagram: https://www.instagram.com/notsabatino