Fireside a series of articles that I’m writing to consolidate my learning.
I have always dreamed of blogging. I have even tried once called 20MinuteRants. They worked quite well as a basic format whereby I can write about things in a fairly efficient manner (hence the 20 minutes), and be able to reflect about the things I’m up to.
The problem with the project is that I rarely had the motivation to do one. Once I was too busy, or out of ideas to write about, I stop. If there’s not anything to rant about, why is there a 20MinuteRant?
Indeed that has been why the blog has been on hiatus for the past many months. I suppose we can consider this entry the last of the 20MinuteRants and the first one of a new series of writings—Fireside—which I hope will continue for a long time.
Impetus
My mentor D has always told me to start arguments with why. The larger system in which Fireside is located, my knowlegebase system, has such a page arguing why its a good idea.
And here’s the why: starting university has made me surprisingly lost in terms of what I want to do. In that: there’s so much of the vicissitudes of daily life that I no longer have the same intellectual curiosity that I think I had during middle and high school. And, in doing this, I hope to get it back.
This illustrates the key goal of Fireside: a once+ per week posting, illustrating some new thing I’m aiming to learn for the week. It can even be in the classroom: but something I’m going above and beyond to try to understand. Each article will either plan something to learn, or summarize my learning in it.
I remember seeing an article through HN at one point (its not this one, it was significantly less “ha ha! business” language, but this will do) that “You never really learn something until you write about it.” A dormmate of mine working on ray tracing also said something to the same effect. And, so, what the heck. Let’s try this out.
A part of me wishes that this fulfills the “deliverable” in my head of “YOU AREN’T DOING ENOUGH!” whenever I spend time wandering aimlessly to try to learn something. If it has become a Fireside, it counts. I guess.
I should also add that Fireside is named Fireside because of FDR’s Fireside Chats, where he got to directly talk to people unfiltered about his views.
Parameters
Frequency
At a minimum once a week. No promises though.
Names
You have already noticed one of the parameters. Unless it is a general concept or a well known thing I did, I won’t be using names throughout the articles. Somewhat contrarily, I firmly believe that the process of building shit is a very personal one. Hence, following the example of one of my favorite essayists Zhu Ziqing, I will be using the first letter of the name I refer to them to refer to all people mentioned. Of course, if you don’t want to be included, I’d be happy to pull things down.
Themes
- technology in general
- random nerdism
- deep learning and language models, methods and applications
- the shit I get up to
We Begin
I hope to begin this weekend. I’ve spent the last while trying to train a serious deep learning model (read: OpenAI Whisper Large V2), and dying because all I have access to is 2 32GB V-100s on the PSC (and yes, I point out that this is terribly privileged statement: woe is me with a cutting edge GPU).
However, the Language Model literally doesn’t fit in the damned box. So, I’m trying to learn about distributed training methods like Ray, methods of efficient tuning with LoRA, and new-fangled memory sharing things like DeepSpeed.
Stay tuned.