Midterms

The last few weeks I've been taking midterms in all of my various classes, and they've been going well for the most part. Not a whole lot of exciting stuff has been going on, but I've been keeping busy.

The most notable thing that's happened recently actually happened on Sunday (two days before the publication of this post). One of the communities I do volunteer development work for started having some really strange issues with their chat bot - the one that I was in charge of. As it had turned out, a broken build had been mistakenly pushed to our image registry, and it wasn't until a few days later when the bot restarted that these changes were pulled and ran. A whole lot of strange side effects started popping up, from having the incorrect status texts, to not even responding to certain commands. At first no one was sure what had happened, as no one knew of any updates that were waiting for deployment. At the time, I didn't have access to the image repository to check any of the images from there, and I was searching through the source code that I had access to try to find any issues - everything was checking out on my end, and eventually we had decided that the issue had to be with a new image that had been pushed without our knowledge.

When we investigated this, we found that a new image had been pushed. We knew that work was being done to push out a set of new updates, but to our knowledge we didn't know that an image had been pushed, so this surprised us all a bit. The image had ended up being configured incorrectly, and it turns out there were a few bugs that hadn't been caught that caused the other issues.

We decided to deploy the last known working image, but due to how our deployments work we needed to build it again - the only issue was our dependencies had changed since we last built this image, and the new build was failing due to those missing dependencies. We had to move up to the most recent stable source code that had not been deployed because of this, and finally got a working image rolled out.

Unfortunately, the source for this reason hadn't been fully tested - while this fixed majority of the issues we were having, some different issues hung around still. Luckily I was able to get some fixes out for those issues, and everything was fully functioning again. This was my first time having to do some real-time downtime mitigation and hotfixing, and it went pretty smoothly from what I could tell. There were some slight issues through the whole process, but in the end everything worked out.

Exploring Campus

Other than mitigating service downtime, I had some fun doing other things this week as well. Most of the day last Friday, I ended up exploring campus with friends. There are a lot of interesting places hidden around different buildings, and we saw some pretty neat things. We already knew there was a utiliduct/utilidoor system all throughout campus, but none of us have the authorization to go in them. While exploring, we did find a number of entrances to the tunnel system, but we were not able to explore them, sadly.

A lot of the more interesting things there were to see were only visible through windows on locked doors. We learned that the campus has two cold rooms - one whole room that can get to -70C (-94F), and another larger room that has smaller chambers. I'm not sure how cold the individual chambers are able to get, but I suspect it may have been around -120C (-184F). I'd like to be able to fact check this, but I'm not sure where I'd start looking for more information on it.

Looking into maintenance rooms and seeing the glycol heating pipe was also really cool. The whole campus is heated using glycol that's piped through the utiliduct system to every building. The glycol is all heated at the power plant that was recently built, and distributed throughout campus.

I don't know much about the infrastructure work that goes on around campus, but I think it would be something I would really enjoy learning about - someone I've talked to that works for campus utilities has shared some information that's really interesting. Through him, I learned that the whole campus has a nuclear fallout plan - complete with water storage plans, counts of how many people campus is able to support (11603 - a number that lines up really well with the number of people currently attending and working here), and even a short list of buildings and labs to potentially avoid due to chemicals and bacteria that may be present there due to experiments. (somewhat concerning is that the shelter is currently only stocked for 4586 of the 11603 shelter spaces, but hopefully we won't need to use them anytime soon)

That's all I've got for this post, but thank you for following along. Exploring campus was a lot of fun, and I'm sure there's even more things that I could find if I looked around more. Until next time, make sure you enjoy yourself.

P.S. I've not gone through any pictures recently, but I'll update this post with some that I've taken soon.