Sat Aug 5 // Sun Aug 6 // Mon Aug 7 – DiffusionBee and More Phantasmagorian Creatures

As I was typing this sentence (on Sunday), something caught my eye from the window. It was a small rabbit, or should I say large bunny, bounding across the lawn. I’m writing this time from the second floor bedroom, on a desk in front of a long rectangular window that allows me to look out over our humble kingdom. From this perch I can gaze out over the yard and – wow, there goes another bunny! That one was not bounding, that was a hurried scamper. A comical scamper. Boy those things can move quick, can’t they. I don’t think that was the same one, I would have noticed it come back across the yard. Same size though. Could be siblings. Could be twins. I guess they’re all kind of like twins, aren’t they, because they all come out at the same time. Twins, triplets, quadruplets. There’s a word for this – littermates. Yes, littermates.

This is extremely stream of consciousness. You’re right along for the ride with me here. I can see all of these things from this window, and more, because I can see the feeder from here. And the lake. I should say, the feeder complex. I have been here for the various stages of this aviation feeding station’s development, and would say that we can now officially call this a complex, the most recent addition being an oval-shaped mulch patch with African Lillies, for the hummingbirds. They like those African Lillies. Here’s a photo, courtesy of the internet, of what they look like.

African Lily – Agapanthus africanus

In the last paragraph, I wrote, “oval-shaped”. When I wrote that sentence, I first wrote ovular, you know, like circular, or rectangular, but it immediately struck me as sus, and my intuition was correct. That word is already taken. For things related to ovules, of course. The English language is weird. The other day we were watching soccer and I said something like, “She’d just shotten the ball” and the parents stopped me and said, “Shotten??” Got, gotten, fine. Shot, shotten, no sir. Gotten is still alive in the common vernacular but doesn’t have to be used (I just got home, I’ve just gotten home), but it might go the same way as shotten, and die out someday. Because, I just did some Googling, it’s not that you can’t say shotten. It’s not incorrect, it’s just a dead word, listed by the dictionaries as obsolete. Once upon time it was used, if we can trust this nice graph from Collin’s Dictionary, some time in the 1700s, and who knows how much before then.

Anyways, back to the African Lillies.. Ours are yellow and orange. They’re dainty things. So now we’ve got some of those below our feeders, of which we have four hanging from two metal poles, that are four feet high or so, and one hanging from a cottonwood next to the mulch oval. From one pole, there are three smaller feeders: one with the sugar water for the hummingbirds, with little fake flowers for them to stick their tiny beaks into, a standard one, we’ll just call it that because I can’t really tell what’s going on with it from this angle, but it looks similar to the feeder hanging from the cottonwood, which has a little ledge in front of it that the birds and the undesirables (the squirrels and the chipmunks) perch on and pull seeds out through a slit in the bottom, and then there is a sack of smaller seeds, with a thin sieve-like mesh skin, that is favored more by birds with skinnier breaks. I’m thinking that the nuthatch might go for this one, and speak of the angel, the nuthatch has just landed. The hummingbird has just shown up as well. It’s a whirlwind out here. At this moment, I can see these birds: a female cardinal, four, then six sparrows, a hummingbird, a nuthatch, a few geese, far off, and some other kind of sparrow, or maybe a chickadee. These guys n’ gals are out here partying every day. Attached to the sack is a small bowl with jelly for the orioles. They were around earlier in the summer, with the red-winged blackbirds. They’ve both gone away now. Hanging from the other pole is a massive multi-storied megafeeder. This is monopolized by the sparrows. There is currently a sparrow at every feeding port, and they’re fighting to keep it that way. The nuthatch keeps trying to get in there. He flies back and forth, looking for an angle, a way in. He finds it, or forces it, gets a few seeds, and is chased off. He’s my favorite of these birds, I have to say. Something about the way he hops and skips, the way he swivels his head, and pulls seeds out of the feeder with his long, sharp beak. He trawls the sides of the cottonwoods, poking and prodding, snapping juicy morsels up out of the cracks, and possibly hiding seeds. I read that birds do that, wedge seeds into the cracks of trees. He’s got a very pretty blue, grey, white, black coloration. A lot of personality in that bird. He could be a she, I actually don’t know. Another hummingbird has just shown up as well. It’s now confirmed that there are two hummingbirds around.

A lot of action going on down there, man. You could watch it all day, especially if you were a cat. From here would be great, but from our downstairs window, a large, three-paned window with a fullscreen view of the feeders. That view is every cat’s dream. Cat heaven. And Daisy heaven is looking at fish. It doesn’t take much, with them. I was sitting out on the deck in the rain yesterday, right under the ledge of the house. I was only being sprinkled on. It was a soft rain, the temperature was cool, but a very comfortable, perfect cool, not chilly, and with low wind. It was just quiet, but not unsettlingly quiet, not dead silent, just quiet, with only the gentle white noise pitter-patter of the drops, on wood, water, and leaves. And with the fresh scent in the air, the fresh scent of earth, of wet wood, of rainwater. Daisy was out with me, laying beside me near the steps, staring off into the distance, out between the large trunks of the cottonwoods, at the geese in the yard. I sat there, watching her, watching the ripples of the water on the surface of the lake, watching the sky, watching the geese, and in that moment, so full of calm, my senses so pleasantly stimulated, a little thought popped into my head, that this was heaven. It was a fleeting thought, really. But it was a solid one. I wasn’t out there for too long before I felt restless, and I didn’t stay. For that short time, though, I guess I had a little taste of it. A brush with the divine. And you know, it really doesn’t take much. It doesn’t take much, to be happy. And it doesn’t have to cost a dime.

Now it’s Monday. Enough talk about the birds and the wind and crap like that. Let’s get down to business.

The text prompt for this image was “Creatures from a phantasmagorical universe, Pastel Art, Beautiful Lighting, Warm Color Palette.” And this image was built in 22 steps. Last post looked at the effect of step count on image generation, and now we’ll talk about the effect of prompt text and seed number. First, the seed number. Like an actual plant, the seed is the basis for the image. How exactly it works I don’t know, but I can tell you that if you use the same seed for an image, even if they come out wildly different in the end because of all of the other parameters, they must have started the same way. So, if you generate an image twice, keeping all parameters the same, including with the same seed, you will have nearly the same image in the end. If you keep all parameters the same and change only the seed, you will have an entirely different image in the end. The seed for that first image, our experiment image, was 54445. Below are images generated with seeds 54446 and 54447, and otherwise the exact same parameters.

Seed: 54446 (Coral reef elephant??)
Seed: 54447

This means that you could download DiffusionBee, set all of the parameters to exactly what I had them as for these images, and you would get nearly the same thing. You don’t get exactly the same thing, because the algorithm that generates these is as they say in the biz, nondeterministic. (Also.. how freakin cool are these pictures. I think I could have a promising career as a Phantasmagorian AI Art Programmer. Wouldn’t that be fun to tell people.) It would be interesting to know what exactly a seed is in the code, how that works. I’m trying to think of what it could be, like a set of numbers or parameters that are related to the growth of the image. I generated three more images with totally different text prompts off of the same seed, to see if that would reveal anything about the seed. 1. “Gorilla in a top hat, by Vincent van Gogh”, 2. “a bowl of cereal, colored pencil, children’s drawing”, and 3. “Barack Obama riding a skateboard, 8-bit”.

Van Gogh Gorillas
Bowls of Cereal
Obamas Riding Skateboards

I can only really see one similarity between them. All of these images have multiples of the subject. I’ve wondered about that, because sometimes there are multiples, and sometimes not, and it doesn’t matter if you specify how many gorillas you want in the prompt text. That may be outside of the prompt’s control, and dependent only on the seed.

Now looking at the effect of prompt text. In the next image, I changed only one thing. In the prompt text, I changed “warm color palette” to “cool color palette”, and now you have an image that is in one way quite different, and yet similar. Take a gander.

Warm Color Palette vs. Cool Color Palette (slide the bar to compare images)

Many differences, and many similarities. You can see that the bones of the image are the same. That’s really where the seed is coming into play. The bones are the same, but the flavor, the details have changed. There is much more of a pronounced glow to the image, which I really love. The whole thing is glowing in mystical blue light. All of the flying fish are gone, and the firecat, the little glowing mushroom lamps, and the red sun in the upper right corner, gone as well. In the cool color palette, you have more detail in the background, less of a foreground (on the sides of the image), and now a really interesting scene at the bottom, with an incredible pink-purple boar creature, and a large, curly, pink monkey. There are new plants, and some yellow thing that my brain is interpreting as a butterfly. Would you expect such a different image just from asking the program to change the color scheme? I didn’t. I thought it would take the same image and just color it differently, but it’s much more than that. I had a lot of fun trying other color schemes and styles and seeing what popped out. Like the chocolates in a box of chocolates, you just don’t know what you’re going to get.

Colorful
Cold Color Palette
Electric
Green Color Palette
Green (without the words “Color Palette”)

They all have the same foundation, but the aesthetic is totally different. So how about changing something else, say, “Watercolor” instead of “Pastel Art”?

Pastel Art -> Watercolor

Amazing. So amazing. Look how the branches of the tree on the bottom right become the hair of the green rhino pokemon creature. The leg of the firecat becomes the leg of the dragon whatever. (I’m trying my best to describe these phantasm creatures to you. It’s hard, ok. I could make up names for them. The Wakkanok, the Schmerkelvitz.) The background just disappears and becomes stars, and the foreground is made of creatures, and colored gas. Now we really are out in the universe. I love it.

Warm Color Palette vs. Cold Color Palette

This one was “creatures in a phantasmagorian universe, Pastel Art, Cool Color Palette” but without “Beautiful Lighting”. That made a huge difference. I’ll take my beautiful lighting, please.

What if we change “universe” to “desert”?

Incredible.

Some of the best, here. On 10 steps, we could more creatures. I love the blurry, dreaminess of the watercolor.

Very cool. I’m really in love with these. You just never know what you’re going to get. So much to play with here, with DiffusionBee. This is a very simple program, no coding required, no importing models or anything. Also, they have AI video now, I’ve seen it. A full movie trailer, 30 seconds live action, apparently made with AI. Think of the implications. We could, potentially, the average person, easily generate hundreds of videos of penguins riding horses. Into battle, at the Kentucky derby, joyously through a meadow, along the beach. This is coming, this is the future. It’s exciting stuff.

Thu Aug 3 // Fri Aug 4 – Hummingbirds, Hummingbird Posers and Diffusion Bees

An incredible thing has just happened. As I sat down on my little table outside, freeing the famous swimming dog to exercise her capacity for infinite joy in her swimming, to write this post, our friendly neighborhood humming flew up to me, two feet in front of my face, at eye level, looked me in the face, and pooped. A tiny white squirt came out of its butt. Now tell me, if that is not blessed, a sign from the divine, what is? It’s that or nothing. The great creator letting me know that it’s a good idea I’ve got, writing this post. This one is for you little hummingbird.

I actually do have a photo of this little birdie, I’m remembering now!

A little blurry because I was shooting through window glass. Sue me. This is the bird. There may be two though. I’m feeling right now like I’ve seen two at this feeder together. Will have to ask the other resident birder (Mom). They like to drink this stuff. Delicious sugar water.

Now this is a great lead-in for the first of our two main topics in this post. A hummingbird-like creature was spotted in the vicinity recently. A creature known in some scientific circles as a Sphingidae.

When you hear the word Sphingidae, what comes into your mind? I’ll give you a minute.


Bing! Times up. Here is the Sphingidae.

If this is your first time seeing one of these creatures, you may be in awe. You may be spectacularly dumbfounded, and I would understand. I certainly was, the first time I saw it. But I saw one out in the wild, outside of my apartment in little old Ozu, in the flower patch with all the cosmos. I stopped to take a goosey gander and my eyes landed on this hummingbird, and the more I started to look at it, the more I started to think, something is wrong with this hummingbird. And I stood there and stared for at least ten minutes, my brain trying its absolute best to comprehend this small, confusing creature that was before me. In all ways it looks like a hummingbird, is a similar size, shape, fluttering about manically, and it moves quick, so you can’t get a good look. I left there not having any idea what it was, but with the feeling that there was something very strange out in the world. I spent a long time wondering what that was until I finally found it in a bug book my neighbor Tamanaga san gave me. In large, beautiful illustration was the hummingbird creature outside of my apartment, and beneath it was written, Sphingidae. (In Japanese, which is ใ‚นใ‚บใƒกใ‚ฌ็ง‘). And the name of the Japanese one, is the Oosukashiba. ใ‚ชใ‚ชใ‚นใ‚ซใ‚ทใƒ. I don’t know what that means. Cephonodes hylas. It’s some kind of moth. Are you shocked? It is a moth that is a hummingbird mimic. I tell you, crazy things are happening in this world.

Oosukashiba – the hawk moth outside of my apartment in Kumamoto

I don’t know what I’ve got in my yard, but it’s not one of these. It doesn’t have the yellow butt. And it has red wings. There can be a lot of variety even within a species though, and between males and females, but this is something else. Apparently the range of the Japanese one is more or less, Asia.

I also saw a nice swallowtail. We all know about those.

So, the next time you see a hummingbird hovering around your flowers.. look closely. Might not be a hummingbird at all. (Might be a Sphingidae.)

Ok, I’ll stop saying Sphingidae. Moving on then. The second topic.

I spent all night last last night making AI art. It’s kind of addicting. We all loved the Picasso AI cats. Let me show you something else.

I’m using DiffusionBee to do this, which is an app that runs off of Stable Diffusion, and is totally free.

This is a gallery of images under the prompt, “creatures in a phantasmagorical universe”. With some extra bells and whistles, like beautiful lighting, cool color palette, and pastel art. DiffusionBee does well with the abstract stuff, like phantasms, and Picasso. In fact, I have a few images of Picasso phantasms as well, as I know you’d like to see.

I personally think that these are stunning works of genius, and if anybody painted this I would think they were a total genius. It is interesting for the art world, because part of what’s so impressive about the work of an artist like Picasso, is the fact that such a thing was able to come out of his brain. That alone, and then you are impressed by the technical skill required to execute the vision. But the real money is in the concept, in the vision. Clearly DiffusionBee has no problem with that. And if somebody just used AI to make an interesting and original artwork, and then simply replicated it in the real world, they would only be using technical skill, and they could just say that it was their idea. Very interesting for the art world, for creators.

Just to show you a little more of what DiffusionBee can do.. creatures in a phantasmagorical desert.

You can see again, DiffusionBee handles abstract works very well. It’s good where something doesn’t have to be perfect, and there’s room for imagination in the work. But something like, “Barack Obama riding a skateboard.” That’s a struggle.

This was the best one, out of twenty. (I do really like this one.) I’ll spare you the others. After this next one.

It was only so long before I wanted to know what was going on under the hood of DiffusionBee, so that I could better control the output. I did some experimenting and learned a bit about how it works, which is pretty fascinating. So, let me tell you about it and then my twilight binge experimenting may have actually done something for humanity.

This is what the app looks like. You type in your prompt, hit generate, and something comes out. You can generate by text, or based off of an existing image, or draw some stuff on top of an image. A few ways to do it.

And here are some of the parameters you can tweak.

About image generation – The image is formed over a series of “steps”. At each step, something is added or taken away from the image. The image is modified in some way, to execute whatever vision the AI has for the image. You will see that the AI builds the image in a very organic manner, I think, that it is not predetermined what the end point could be, but it is literally created over a series of steps. Let me show you what I mean.

This image is our starting point. It was the basis for much experimentation. The exact parameters and prompt are:

Seed : 54447 | Scale : 16.95 | Steps : 22 | Img Width : 896 | Img Height : 896 | Negative Prompt : human, person | model_version : 1.5 | Sampler : ddim | Similar Imgs : No
Prompt: “creatures in a phantasmagorical universe, Warm Color Palette, Beautiful Lighting, Pastel Art”

The maximum number of steps is 75. This image took 22 steps to make. I used the exact same settings, changing only the step count, to see what was happening along the way, and what effect the number of steps was really having on an image. I can’t figure out how to add captions (lame). The sequence is 1 2 3 4 5 6 7 10 22 30 50 75 (number of steps). Take a gander.

It’s pretty incredible right? Shape, structure, life formed out of primordial ooze. Just like how the universe as we know it was created. Some of my thoughts here.. error on step 3, don’t know about that error. The image is gradually defined across the steps, but the amount of change seems to vary drastically. It would be interesting to know how exactly DiffusionBee determines how much work to put into a step. I would imagine it was determined by some standard metric, maybe time, or amount of data. The unit of generation is s/it, so possibly determined by a set number of iterations? If that is short for seconds per iteration. The difference between steps 6 and 7 is massive, and the difference between steps 22 and 75 are really minute. The image is pretty much fully formed at around 22 steps, and any more, the program just doesn’t really know to do, because it’s basically done. This is good to know when generating these, because it takes about 5 minutes for my computer (a powerful Macbook Pro, 16 GB of Ram, M1 processor) to make this image with 75 steps, and only about 2 minutes at 22 steps. 10 steps was maybe 30 seconds. The image is quite different along the way, at the earlier steps being smoother, wispier, and even with totally different content. At step 10 the subject creature has a trunk, and even has an eye. As the image evolves, that creature is lost to the less-interesting fire cat. Sorry firecat. There is also the whole manta ray-like creature, looming up above, that degenerates into the background. So, you can have a totally different image from one step to another. I saw this again, it was really shocking, in the following sequence of images. These are images of steps 6, 10, and 12, of a different prompt of phantasmagorical creatures. At 6 steps, no creatures, at 10 steps, BOOM, so many creatures! and then at 12, gone again. Blink and you’ll miss them! Truly phantasmic creatures. So if you generate this image on any settings but 10 steps, you’d think your prompt failed and you came up empty, when it isn’t true. This series in particular really left me feeling that 10 steps was a magic number, along with 6 and 22.

So.. this is the effect that step number has on an image. Based on the step count, you can have vastly different images. More steps is not necessarily better and can even be less desirable. I didn’t only play with step count. I played with seed number, and the effect of words in the text prompt. My eyes are tired of all this squinting, since I’m writing outside, and my post is starting to lag for whatever reason, so I’ll save that for the next post. Arigato robotos, and more DiffusionBee talk next post. ๐Ÿ™‚