MoreHavoc

The WhisperFrame Generates Art From Conversations

2023-09-17T23:55:45+00:00

The WhisperFrame

The WhisperFrame listens to conversations in our living room and then generates art based on those conversations. It’s a pretty simple concept:

Record snippets of the conversations
Convert those to text via OpenAI’s Whisper API
Combine recent transcripts together and have GPT-4 pick a topic and generate an image prompt
Use Stable Diffusion to generate an image from the prompt
Show it in the living room!

Ours generates a new image after every 5 minutes of active conversation. When there hasn’t been any talking, it will revert to showing randomly selected images generated in the past.

Check out the images and prompts (658 of them) it has generated from our living room.

In general, the moving parts are:

A loop that records 15-20 seconds of audio, then sends it to the OpenAI Whisper API to get a transcript. And repeat. Every 5 minutes or so of recorded material, go to the next step.
Take the transcript and use GPT-4 to extract a topic as an image prompt. We manually created some outputs to give it a couple of examples to go by.
Send that prompt to Stable Diffusion to get an image. When done, return to step 1.

A separate Flask server runs in the background, serving up the current image, and every 5 minutes, checks again to see if there is a new image. If there hasn’t been a new image in 30 minutes, it reverts to just showing random pictures from the list until a new one is generated.

This helps keep the content fresh and relevant to the conversation in the room.

It’s a bit self-fulfilling in that as people talk about the image it drew, it becomes more likely that it tries to illustrate that one again, as the topic is more likely to be selected by GPT-4. But it’s still awesome!

It was a fun project; I even created a second one for my office that generates images during meetings! It might even be a new way to make meeting notes, a list of images representing the meeting instead of action items. It probably won’t catch on, though!

Using AI to Protect Web Form Inputs: Extending Beyond the Adopt-A-Drain Example

2023-05-10T12:30:45+00:00

Note: This post is a brief summary of a video I made about using AI to keep names clean and safe in adopt-a-drain programs. I’ll update this post when the video is available. This post was made by feeding the transcript of the video into GPT-4 and asking it to write a blog post. It was then edited by me.

Artificial Intelligence (AI) has become an integral part of our daily lives, from setting timers (via tools like Alexa) to writing code. One particular use of AI that I find interesting is its application in protecting web form inputs. In this blog post, we explore how GPT-4 AI can be used to safeguard input forms across various domains, using the “adopt-a-drain” program as just one example.

Web forms often require users to submit information, ranging from names and email addresses to more sensitive data such as credit card numbers. Ensuring that the submitted information is appropriate and valid is of utmost importance. For instance, in an Adopt-A-Drain program, users may submit creative names for drains that could be inappropriate or offensive. We can use GPT-4 to check those names when the user submits them.

GPT-4 AI can be used in chat mode to review the submitted inputs, regardless of the specific use case. By crafting a prompt that outlines the AI’s role in assessing the appropriateness or validity of the submitted information, we can instruct the AI to provide feedback on whether the input passes or fails the required criteria. If the AI is uncertain, it can flag the input for review by a human moderator.

One of the significant advantages of using AI like GPT-4 is its ability to understand unstructured prompts. It can interpret instructions and provide responses in a conversational manner, making it easier to engage and interact with the AI. Furthermore, GPT-4 can be “programmed”¹ to respond in different tones, such as professional, funny, snarky, or sarcastic, depending on the desired outcome.

In addition to using GPT-4 AI, the OpenAI API offers a moderation endpoint, which acts as a first line of defense against inappropriate inputs. By categorizing the input, the moderation endpoint helps protect the system from offensive or invalid content. If the input passes the moderation endpoint, the more powerful GPT-4 AI can then be used to assess the input’s suitability or validity.

In conclusion, AI like GPT-4 offers a powerful and adaptable solution to protect web form inputs across various domains. By crafting the right prompts and utilizing AI’s conversational abilities, we can create systems that ensure appropriate and valid content is submitted. The Adopt-A-Drain example is just one instance demonstrating the potential of AI in safeguarding web form inputs, and its applications can be extended to numerous other scenarios.

Footnotes

1 - I put programmed in quotes because you are really just telling it, like you would a human doing the same work.

OctoPrint Saves Me From a Blue Screen of Death!

2023-05-10T12:30:45+00:00

OctoPrint Saves Me From a Blue Screen of Death!

Recently my trusty Lulzbot Mini 3D Printer was having issues.. well actually the old Windows laptop I used to run it was having issues. When I set out to replace the printer I stumbled upon OctoPrint, which not only saved my printer, but made it much better than it ever was before.

This is that story:

A few more details for those that want to try this on their own

I got the basic tutorial from here (I know instructables gets a bad wrap sometimes, but this was a great starting point!)
An OctoPrint deploy script that made thigs much easier
The case I used, but only the bottom half
Getting started with the Libre Potato board
I got my Libre board on Amazon.
Make sure you have a sufficient power supply otherwise the board might brownout while printing

Meet Portia, The Candy Spewing Spider!

2023-04-17T23:55:45+00:00

I finally finished making my video of Portia!

It was a fun project that mostly worked in the end. I also enjoyed making the video so, more to come in both departments!

Building a Needle Container

2023-01-30T12:30:45+00:00

I’m pretty into making custom things with my 3D printer these days. So when a random request comes in from a friend like:

Hey, this little plastic tube I used to hold darning neadles broke, can you make me a new one?

I get excited. So I made a quick hexagonal tube, with a threaded lid!

The lid is a little too tall, so has too many threads. The number of times you turn it to remove the top is a little silly, but it works! Next time I would aim for just a fully turn to unscrew the lid.

Pictures

STL Files

The Needle Holder

The Cap

Building a Swift

2023-01-21T23:55:45+00:00

My friend, Bee, does a lot of knitting. She recently was debating if she should buy a swift to help winding balls of yarn. They are pretty cheap online, but I wanted to see if I could make one, so here comes an Umbrella Swift!

Research

I first looked at pictures of swifts, like these:

Then I browsed Amazon. The basic design hasn’t changed much, so I set about building a simple Umbrella Swift.

The Process

I started by designing the top cap and the bottom of the umbrella in Fusion 360 and printing them on my 3D Printer.

The top of the swift

The bottom of the swift

Yes, they are all hexagons! Partially because this is Bee we are talking about here, and also because hexagons are the bestagaons.

The Clamp

I had never designed or printed threads until this project, but I really wanted the clamp on the bottom to be 3D printed as well. Turns out it was Super Easy, Barely and Inconvenience! Really, I just used the Thread tool in Fusion 360, set it to “Modeled” and then just pushed back each of the three surfaces of the thread by 0.1 mm.

Construction

I used a hardwood dowel as the shaft, and 15 inch bamboo “craft sticks” as all of the bars. A few random nuts and bolts here and there, some locktite, and of course my old friend hot glue to hold the top onto the shaft.

Final Product

Download STL files:

NFTs from a Camera!

2023-01-10T23:55:45+00:00

NFTs

I don’t really want to cover what an NFT is here in this post, there are lots of places to read about that, like here, or here, or here is a funny story about some NFTs…

What you need to know for this idea to make sense is that an NFT is a way to make a digital thing unique. If you make a painting, there is only one original. If you take a picture on film, that’s your original. It’s uniqueness gives it some of it’s value.

With an NFT, you can make a digital asset unique. You can then sell it as though it was the only one.

AI

One other thing… AI image tools have gotten pretty good and are only getting better. I can feed prompt like this into Dall-E:

a photograph of the Eiffel tower taken across the seine with the champ de mars in the background and a beautiful sunset behind it.

And get this, slightly impossible picture:

Why can't this picture be real? (other than the obvious funny business?)

This should be taken from roughly the north side of the Eiffel tower, with a sunset to the south, which shouldn't be possible.

How do we know photographs are real? How do we know they were really taken by a human, and at the real place?

NFTs direct from a Camera

The idea here is that you have a physical device that let’s you take pictures. This device is not your phone, we’ll call it a Camera for lack of a better word. When you take a picture with this device you can preview it, and if you approve it, and if the camera has a connection to the internet, it will:

Write it directly to IPFS and get it’s global URI back
Write the global URI to the etherium ledger as belonging to the owner of the camera.

A few notes:

The image is never written to disk anwyere, it it stored in RAM and then written directly to IPFS.
The original image is copied. If you want to edit it, you can get it from IPFS and use it as you normally would.
There could be a whole set of additions to photo editing software to handle the chain of changes for an image. As you work on it, the new one is written to IPFS and marked on the ledger as yours, and being relative to it’s parent image. This keeps a chain of custody for the image as you touch it up.

The project

It might be fun to build a simple version of this. Perhaps using a Circut Python board, a simple serial enabled camera and perhaps a nice case to put it all in.

It even would have the added benefit of being pretty low resolution, really bringing back that old digital camera feel!

;-)