Can I vibe?

Testing out the efficacy of vibe coding

Recently my wife asked me to put together a Discord server for our school community. As part of that, I thought it would be cool to have a Discord Bot that could respond to different questions and point the members to different resources. I’ve been kind of looking for a project to try and dust off some of my coding skills and this seemed simple enough to give it a shot. But instead of building it from scratch, why not use AI to do most of the work?? That’s the future, right? I want to be part of the future. So I tried out a few different services to see whether they could do the heavy lifting for me.

I have no CS degree, but I did do a fair amount of coding after grad school and for a year (in between other jobs) I was employed as a Senior Software Engineer. My language of choice was Actionscript 3, but I also know a little Python, Javascript, etc… I have not done any coding in a professional capacity since 2018, and I have not done it full time since 2012. And FWIW, even when I was doing it full time, I was not “good” at it. Mostly I hacked together other people’s code and made a big mess.

I tried out three services to build my Discord bot - Replit, Cursor AI, and ChatGPT. Here’s how it went:

Replit

1. I started with Replit because i liked the idea of having them host the bot as well as build the bot. I used the starter edition which gave me 10 checkpoints (whatever those are). I asked the agent to “build me a Discord bot in Python”, sort of figuring it would start slow and then check in. It felt like when you drive a car and you aren’t used to the sensitivity of the gas pedal, so you give it a little pressure and find yourself careening towards things. The Replit agent took off, building out a whole bunch of files and directories, while telling me what it was doing, but at a pace that I couldn’t keep up with. It did give me helpful information about what i had to go do on the Discord Developer Portal, and it asked me for my token and presumably stored it somewhere safe, though I never checked on that. At the end it told me the bot was running. Great start!

2. Then I asked Replit to build me a feature that would let the bot respond to messages that were directed at it. I figured this was pretty simple. The agent took off again, changed a bunch of files, and came back saying that it was ready to go. I tested it and was able to send a message to the bot. But… for some reason commands weren’t working. Aaaaand… i had never actually checked to see if they had worked back in step 1. So, yeah. I did a little old school “googling”, and found a crusty old stack overflow article that suggested perhaps in creating a custom message intent I had clobbered the commands handler. Which was funny becuase there was a comment in the Python code that basically said: “we don’t need to worry about commands, they’re processed elsewhere”.

3. So I politely asked the agent whether it had indeed clobbered the commands handler. It told me that in fact it had done that, which was definitely a mistake, and it was going to go fix that right now. So it took off again, changed a bunch of files, and came back and announced that it had succeeded. THIS time i was smart enough to test it. And it worked! But… for some reason, every command or message I sent got anywhere from 2 to 4 replies. And I don’t know a ton about Discord, but i know the bot isn’t supposed to spam reply you. I figured this was either some error in the code, or maybe i had multiple instances running somehow? I admit i’m not super clear how Replit works so maybe there was a new version of the bot built every time I made a change? Anyways, I figured the best thing to do would be to stop all instances of the bot and start them over. I went to click the “stop” button but there was a warning message telling me that if I was working with the agent and stopped the process, bad things could happen. Well, ok.

4. So I asked the agent to stop the process. This was interesting because the agent didn’t exactly do what I asked. Instead it said that it noticed a different issue that was causing the duplication of messages, and took off in a frenzy to fix it. After that, it then tried to stop all instances of the bot running. This was humorous because as best i could follow along, it kept trying to stop something called “gunicorn” and it wasn’t working, so it repeatedly increased the intensity of the commands. This led to my favorite part of this entire project, as the agent was trying to use the “killall” command, to which the console reported “killall command not found” and the agent reporting that it was successful. And at no point is it asking me my opinion on any of this, it’s just off to the races, and I’m watching hoping that it doesn’t decide the next step is to burn down the data center. I also learned that anytime I asked it to do something, it creates a checkpoint, even if it’s fixing something that it broke by doing something else. And at least with this version of the plan, i only get 10 checkpoints, so at this point i’m very motivated to maybe not ask the agent to do too much more.

5. Luckily, when it finally was able to defeat the stubborn gunicorn, it stopped changing things and trying to stop things and created one more checkpoint. At this point, the bot responds to messages with a very basic “hello” and can answer all its commands. It only does it once and it’s live and running. Whew!

Cursor.AI

1. I downloaded the Cursor AI IDE and installed it on my mac. I told it what I wanted to do, and it built out a few different files and structures locally on my laptop. One issue that came up is that when it gave me instructions for how to run the project locally on my machine, it turned out that I didn’t have the right python version hooked up to the “python” command in my shell (it was “python3” instead). I eventually figured this out but not before trying to add a few different paths to my PATH variable, which Cursor was helpful with, but explained it as if it was very normal for me to just edit my PATH variable all the time.

2. I ran into the same issue as above where the code to respond to messages clobbered the code that was handling the commands in the bot. Since I had already been through it once, it was much easier to spot and fix (introducing some bias into the comparison), but I was surprised that the default for all these seems to be overriding a function that then clobbers key functions.

3. Style wise, I found the code python code that Cursor was writing to be more “human readable”, at least to me. The functions and variables were well named and even with a very rusty background, I was able to at least guess at what each function in each file was doing.

4. My biggest challenge came near then end where the bot was installed but not responding to anything, and this ended up being becuase I had to fully remove the previous bot that I created and then make a new one in the Discord environment, giving it the same permissions. Cursor suggested this course of action, but only after I asked it whether or not it was necessary.

ChatGPT

1. I asked ChatGPT to build me a bot which could reply to messages and execute some commands. The biggest difference is that ChatGPT is not an IDE, so it took the approach of giving me step by step instructions on what to do, assuming I was starting from nothing. It walked me through how to install the correct Python extensions on my machine, and told me how to go to Discord and set the right permissions for everything before it gave me any code.

2. It generated probably the most simple code of all the interfaces, to the point where it had a single file that did all the work (the others had a few different files). One caveat with this is that it put the public key directly in the file it told me to create, which is fine for running locally but i’m like 75% sure it shouldn’t be there if I was to host it publicly or if I ever decided to store the file on Github.

3. Other than the extra copy/paste step, it probably did the best job of asking me clarifying questions and presenting the information in a manner that I could easily understand as a novice.

4. It worked on the first try, but at this point having done this three times, I was ready for any hiccups.

5. Then it asked me whether I wanted to host it, and even suggested using Replit to host alongside a workaround that fakes 100% uptime. \

Overall

Had I to do this again, i would have STARTED with ChatGPT, which seemed much more geared towards novices - though I suppose there’s a decent chance this would have ended with my uploading my private key to the internet. Unclear.

The actual coding part was less complex than all the setup and configuration, and if I hadn’t reached back in my brain for how some of those things worked, I”m not sure I would have made it through. Seems like many of even the “easy” steps still involve opening a terminal window and futzing around with things the average person has likely never done.

Outside of hosting on my local machine, the bridge between “build it to test it locally” and “get it running on a server” appears to still have gaps that require some degree of domain knowledge.