nfraprado

Favorite Game: Hyperbeat

2026-04-07T00:00:00-03:00

Every once in a while you play a game that is unlike any other, that transcends genres, that you can feel the author speaking through it to you, and that you can tell will stick around in your mind in some way or another probably forever. Hyperbeat is one of those games.

I love how the way to interact with anything in the hub is to just wait while a musical rest fills in, I love the 1 FPS walking animation, I love how philosophical the entire narrative and dialogues are, and how the different songs are used to emphasize feelings in the dialogue. And I LOVE the soundtrack.

The gameplay is hard, yes, and I missed many beats even in the tutorial. But you can clearly feel yourself improving as you play, and it's very rewarding.

This is how my character looked by the end of the game:

Thank you to Noclip2's Computer Worlds video for featuring this game, otherwise I might have never experienced it.

Now if you'll excuse I'm going out for a walk, because I feel excited and I can't sit still while listening to the Hyperbeat soundtrack.

2025 retrospective with shirts

2025-12-29T00:00:00-03:00

This year I bought more shirts than ever. Mostly because I decided to go to as many concerts as I possibly could.

With the year coming to a close, I thought this would be a good moment to look back at the shirts I collected throughout the year and see the story that they tell. So here it is, my 2025 retrospective through shirts:

From top left to bottom right:

Row 1, Column 1: March. Went to a Three Days Grace and Disturbed concert. Three Days Grace was my favorite band as a teenager, and on the day of the concert I found out that Adam Gontier, the lead singer, who had left the band several years before, had just joined back. That show was a dream come true for my inner teenager.
1-2: March. I saw someone wearing this shirt in a picture of a local volunteering group I was part of and immediately asked where I could buy one. In fact I bought two extra to gift them to friends. It has been my most popular shirt by far, with many compliments received on different occasions and someone once even asked me if they could take a picture of it. If you also love it, check it out here: https://mtpfriends.bigcartel.com/product/what-s-more-punk-adult-t-shirt
1-3: April. Went to a social event in the neighborhood, and saw a local, and pretty good, band called Lolux.
1-4: May. Went to a Bloc Party concert at the Forest Hills Stadium. I came out of it feeling underwhelmed by the sound quality but it was fun nevertheless. Plus it made me discover Forest Hills, which I found a very quaint neighborhood that is hard to believe exists in New York City.
2-1: June. Wonderville is simply put my favorite place in NYC. As they put it, they're "Brooklyn’s home for independently made arcade games". The place looks very much cyberpunk-y, the games are all free to play, and they frequently have unique events, from livecoding to game jams to beyblade tournaments. This shirt was made to commemorate their 6 year anninversary, which I pre-ordered but ended up not making it to the party. I grabbed the shirt a few weeks later when I went there to show the place to a friend that was visiting NYC, something that has become a tradition for me (this is the third visiting friend I bring there).
2-2: June. Went to a Godspeed You! Black Emperor concert in Norwalk, CT. The 1-hour train ride to a small town definitely added to the vibe. The concert was really unique, featuring a stage that was almost completely dark with hand-crafted film projections showing behind, and a sound that would slowly build up from a lonely silence to an overwhelming array of layered distorted guitars and violins. Listen to Sleep to get a glimpse of it, but seeing it live was something else.
2-3: July. Went to a live Welcome to Night Vale show in Brooklyn. I used to listen to this podcast while in college, so it was quite nostalgic to hear it again, and seeing it live, with crowd interactions made it even more fun.
2-4: July. Went to see Creed at the Jones Beach Theater. Beautiful outdoor venue facing the sea, and it was awesome to hear them live. This was the first concert that I went with my mom and the fact that we took a train there and stayed at a hotel until the next day made it feel like a short vacation trip.
3-1: July. I love post-rock, and small concerts, so even though I had only listened to a single album from We Lost the Sea many years before, I couldn't pass on the opportunity to see them for just 25$. It was a blast, those cheap small concerts are the best.
3-2: August. In contrast, shortly after I payed 171$, the most expensive tickets of my life, to see Linkin Park at the Prudential Center in Newark. The view was good, but my seat was so far high that I felt detached from the concert. Also despite getting this shirt in the same size I always get them, this one is absolutely huge.
3-3: August. Went again to the Forest Hills Stadium, this time to see The Black Keys with a friend. It started raining heavily (ironically given the shirt text) shortly after they started playing, but luckily it stopped in time for the show to resume, despite being a bit shorter.
3-4: September. Went to see Breaking Benjamin and Three Days Grace (again!). Despite being my favorite band as a teen, seeing TDG this second time was much worse, and gave me the feeling it's not worth it to see a band twice. They played pretty much all the same songs, and this time they had a big screen behind showing clumsy visuals that were clearly AI-generated. Thankfully Breaking Benjamin was fantastic and made it worth it.
4-1: October. The night before heading out on a trip, I saw Anamanaguchi at the Brooklyn Steel. I only knew two of their songs, of which they only played one, but I really liked their new album that I first heard at their concert. They also played Hopes and Dreams from Undertale, to celebrate the creator's birthday, and that made this a unique and unforgettable experience.
4-2: October. I spent the largest part of October on a road trip throughout California with a friend as I mentioned in the "Song of the Month: Christopher Cross - Sailing" post. There were many natural wonders, but my favorite were the sequoias at Sequoia National Park. That's why I had to stop at the gift shop at the park and get a shirt (and pins!) as mementos of this wonderful place.
4-3: November. And to finish off the year, saw Of Monsters and Men at the Brooklyn Paramount with my girlfriend. The pit was pretty busy so it was hard to see at times, but the venue was really gorgeous and we had a good time. This shirt is a long sleeve, the first one I got in over ten years, and it's so pretty. I think it's my favorite shirt of the year.

Song of the Month: Andy Williams - Do You Hear What I Hear?

2025-12-01T00:00:00-03:00

Slightly late for November, but better late than never!

As November rolled by, and the Christmas mood took over, I put on an old school Christmas songs playlist on YouTube.

Many of the songs I was hearing for the first time. A few of my favorites were Carol of the Bells, The First Noel and Darlene Love - Christmas (Baby Please Come Home).

But my favorite of all has to be "Do You Hear What I Hear?", particularly the version sung by Andy Williams, so that's my pick for the month:

Andy Williams - Do You Hear What I Hear?

Song of the Month: Christopher Cross - Sailing

2025-10-28T00:00:00-03:00

This month I went on a two week long road trip through California with a friend. We sailed on Lake Tahoe, walked over sand dunes in Death Valley, and hiked between the mountains in Yosemite Valley. We saw the oldest trees on Earth at the Ancient Bristlecone Pine Forest, and also the largest trees, at Sequoia National Park, through a mysterious fog.

We listened to many songs throughout the trip, but there was one that popped up and seemingly immediately won my friend over, as he kept putting it back on time and again. It is indeed a beautiful song, and will forever carry the memory of this amazing trip. It is my pick for this month:

Christopher Cross - Sailing

Song of the Month: Moin - Lift You

2025-09-30T00:00:00-03:00

My pick for song of the month this time around is a track that played at Spectacle Theater before the movie started (the movie was Mirage Men (2013), by the way). The lyrics are very poetic, and the instrumentation has this eerie and underground feel to it. It's a great song on its own, but it was elevated by being played at one of my favorite places in NYC and perfectly matching the vibe.

Check it out:

Moin - Lift You

Pin collection update (128 pins in three images and a table)

2025-09-04T00:00:00-03:00

It's been almost two years since I posted the "Collecting pins" post! It's time I shared an update.

Somewhere along the way I decided to get more organized and started a spreadsheet to track all the details about my pin collection (has this crossed the line to the unhealthy hobby territory yet?). The main thing I wanted to have was a clear history of the collection: when, where and how was each pin added to the collection. And after spending a lot of time filling in the blanks, today I can finally share it!

So here it is, my current pin collection, with 128 pins and filling up two and a half display cases, summarized in three images and a table:

No.	Name	When	Where	How
1	Poring — Ragnarok Online	2022-12-01	Online	Purchase
2	Poporing — Ragnarok Online	2022-12-01	Online	Purchase
3	/Gg Emote — Ragnarok Online	2022-12-16	Online	Purchase
4	Deviruchi — Ragnarok Online	2022-12-16	Online	Purchase
5	Fairy Bottle — Zelda	2022-12-16	Online	Purchase
6	Eevee — Pokémon	2022-12-21	Online	Purchase
7	L — Death Note	2023-02-08	Geek store in Antwerp, Belgium	Purchase
8	Phantom Of The Opera	2023-02-13	Phantom of the Opera show at Majestic Theatre in NYC	Purchase
9	Van Gogh's The Starry Night	2023-03-11	MoMA in NYC	Purchase
10	King — Katamari Damacy	2023-03-23	Fangamer stand at PAX East 23	Purchase
11	Prince & Katamari — Katamari Damacy	2023-03-23	Fangamer stand at PAX East 23	Purchase
12	Annoying Dog — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
13	Flowey — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
14	Sans — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
15	Frisk — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
16	Papyrus — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
17	Toriel — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
18	Undyne — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
19	Mercy Button — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
20	Fight Button — Undertale	2023-03-23	Fangamer stand at PAX East 23	Purchase
21	Junimo Package — Stardew Valley	2023-03-23	Fangamer stand at PAX East 23	Purchase
22	? Block — Super Mario Bros.	2023-03-23	PAX merch stand at PAX East 23	Purchase
23	Mario — Super Mario Bros.	2023-03-23	PAX merch stand at PAX East 23	Purchase
24	East 2023 Logo	2023-03-23	PAX merch stand at PAX East 23	Purchase
25	PAX East VHS	2023-03-23	PAX merch stand at PAX East 23	Purchase
26	Babaa — Neopets	2023-03-24	Geekify stand at PAX East 23	Purchase
27	Wraith Paintbrush — Neopets	2023-03-24	Geekify stand at PAX East 23	Purchase
28	Strawberry Fields Forever Paintbrush — Neopets	2023-03-24	Geekify stand at PAX East 23	Purchase
29	Raz — Psychonauts	2023-03-24..26	Trading hall at PAX East 23	Gift
30	Kerbal Space Program	2023-03-24..26	Trading hall at PAX East 23	Gift
31	Atlas And P-Body — Portal 2	2023-03-24..26	Trading hall at PAX East 23	Trade
32	Diamond Ore Block — Minecraft	2023-03-24..26	Trading hall at PAX East 23	Trade
33	Mae — Night In The Woods	2023-03-24..26	Finji stand at PAX East 23	Purchase
34	Green Flame — Acquisitions Inc.	2023-03-24..26	Acquisitions Incorporated stand at PAX East 23	Purchase
35	Psychokinetic Raz — Psychonauts 2	2023-03-24..26	Fangamer stand at PAX East 23	Purchase
36	Goose — Kickstarter	2023-03-24..26	Kickstarter stand at PAX East 23	Reward
37	PAX Gold Mixtape	2023-03-24..26	Carnival wheel at Pinny Arcade booth at PAX East 23	Reward
38	Portuguese Tile	2023-05-06..07	Portugal (Porto?)	Purchase
39	Software Freedom Conservancy's Logo	2023-05-08..11	Collabora Meetup in Portugal	Gift (from Padovan)
40	Collabora's Logo	2023-05-08..11	Collabora Meetup in Portugal	Gift (from Guy)
41	+Finely-Crafted Dwarven Pin+ — Dwarf Fortress	2023-05-20	Online	Purchase
42	Fairly Oddparents	2023-06-06	Online	Purchase
43	Loto	2023-11-09..11	Home in NYC	Gift (from Tony)
44	Ecuador	2023-11-09..11	Home in NYC	Gift (from Tony)
45	Los Andes	2023-11-09..11	Home in NYC	Gift (from Tony)
46	Canada Wilderness	2023-11-09..11	CN Tower in Toronto / Home in NYC	Gift (from Tony)
47	Paranaguá Train	2023-11-09..11	Home in NYC	Gift (from Tony)
48	Wonderville Arcade Machine	2023-11-10	Wonderville in NYC	Purchase
49	Mask — Mr. Robot	2023-11-15	Online	Purchase
50	Squidward At Work — Spongebob Squarepants	2023-12-01	Home in NYC	Gift (from Mom)
51	Penguin	2023-12-01	Home in NYC	Gift (from Mom)
52	Outback @	2023-12-03 (originally ~2010)	Home in São Paulo	Gift (from Mom)
53	Outback Steakhouse Sign	2023-12-03 (originally ~2010)	Home in São Paulo	Gift (from Mom)
54	Paper Boat	2023-12-09..15	Cruise to northeast of Brazil	Purchase
55	Magic The Gathering Card Back	2023-12-15..2024-01-27	SoGo Plaza Shopping in São Paulo	Purchase
56	Odish — Pokémon	2023-12-15..2024-01-27	SoGo Plaza Shopping in São Paulo	Purchase
57	This Is Fine (First Panel)	2024-05-29	Online	Purchase
58	All Hail The Glow Cloud — Welcome to Night Vale	2024-05-29	Online	Purchase
59	Spelcheck	2024-05-29	Online	Purchase
60	Donut	2024-06-01..2024-08-01	Gift shop at Bryant Park	Purchase
61	Schroeder — Peanuts	2024-06-01..2024-08-01	Newtown HQ store in NYC	Purchase
62	New York Pidgeon	2024-06-01..2024-08-01	Newtown HQ store in NYC	Purchase
63	New York Sewer	2024-06-01..2024-08-01	Newtown HQ store in NYC	Purchase
64	Groundon — Pokémon	2024-06-01..2024-08-01	Say Cheese store in NYC	Purchase
65	Pikachu — Pokémon	2024-06-01..2024-08-01	Say Cheese store in NYC	Purchase
66	MTA Logo	2024-08-16	New York Transit Museum	Purchase
67	Dollyinho	2024-09-21	Vienna, Austria	Gift (from Tony)
68	Imagination! — Spongebob Squarepants	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
69	Mr. Krabs' World's Smallest Violin — Spongebob Squarepants	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
70	Nail in Head Patrick — Spongebob Squarepants	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
71	Shocked Patrick — Spongebob Squarepants	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
72	Speedy Sonic — Classic Sonic The Hedgehog	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
73	Pizza Slurpin' Gir — Invader Zim	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
74	Aang On Air Scooter — Avatar: The Last Airbender	2024-10-17	Zen Monkey Studios stand at New York Comic Con 2024	Purchase
75	Mr. Incredible — The Incredibles	2024-10-17	Disney pin trading event at New York Comic Con 2024	Trade
76	Woody, Buzz Lightyear & Jessie — Toy Story	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
77	Randall Boggs with Scream Canister — Monsters Inc.	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
78	The One Ring — The Lord of the Rings	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
79	Pacman	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
80	Chicken Little	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
81	Warp Star Kirby	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
82	Kirby	2024-10-17	Pins stand at New York Comic Con 2024	Purchase
83	Coraline	2024-10-17	Spooksieboo stand at New York Comic Con 2024	Purchase
84	BFF Frog	2024-10-17	NYCC merch stand at New York Comic Con 2024	Purchase
85	BFF Rat	2024-10-17	NYCC merch stand at New York Comic Con 2024	Purchase
86	NYCC	2024-10-17	NYCC merch stand at New York Comic Con 2024	Purchase
87	Ouroboros — Fullmetal Alchemist	2024-10-26	Online	Purchase
88	Greg — Over The Garden Wall	2024-10-28	Online	Purchase
89	Spectacle Logo	2024-11-02	Spectacle Theater in NYC	Purchase
90	Crash Bandicoot	2024-12-10	Uber ride headed to trampoline park in São Paulo	Gift (from Marianna)
91	Black Cat In Bucket	2024-12-10	Uber ride headed to trampoline park in São Paulo	Gift (from Marianna)
92	Zé Gotinha	2025-01-12	Tony’s home	Gift (from Tony)
93	Tudor Rose	2025-01-18	Tony’s home	Gift (from Tony)
94	Irmão Do Jorel	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
95	Coco Mágico — Irmão do Jorel	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
96	Mônica 1963 — Turma da Mônica	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
97	Sansão — Turma da Mônica	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
98	Ratinho — Castelo Rá Tim Bum	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
99	Senta Que Lá Vem História — Castelo Rá Tim Bum	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
100	Minhocão	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
101	Liberdade	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
102	Cavaquinho	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
103	São Jorge	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
104	Capivara	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
105	Piso de Caquinho	2025-01-20	Santa Hell store in Galeria do Rock in São Paulo	Purchase
106	Lucky Cat	2025-01-30	Online	Purchase
107	Andorinha	2025-01-30	Online	Purchase
108	Fliperama	2025-01-30	Online	Purchase
109	Metrô SP	2025-01-30	Online	Purchase
110	Rua Augusta	2025-01-30	Online	Purchase
111	Bandeira Brasil	2025-01-30	Online	Purchase
112	Tv Cultura	2025-02-02	Street fair close to Av. Paulista in São Paulo	Purchase
113	Globo	2025-02-02	Street fair close to Av. Paulista in São Paulo	Purchase
114	Linux Conectiva	2025-02-02	Street fair close to Av. Paulista in São Paulo	Purchase
115	Windows 2000	2025-02-02	Street fair close to Av. Paulista in São Paulo	Purchase
116	ABNT	2025-02-02	Street fair close to Av. Paulista in São Paulo	Purchase
117	Daft Punk	2025-03-18	Online	Purchase
118	Chrono Trigger	2025-03-27	Online	Purchase
119	Plumbob — The Sims	2025-03-28	Online	Purchase
120	Cardcaptor Sealing Key — Cardcaptor Sakura	2025-03-28	Online	Purchase
121	Triforce — Zelda	2025-03-28	Online	Purchase
122	Akatsuki — Naruto	2025-03-28	Online	Purchase
123	Klonoa	2025-03-28	Online	Purchase
124	Love From Wilmot — Wilmot’s Warehouse	2025-04-26	Online	Purchase
125	Usual June	2025-04-26	Online	Purchase
126	Hope Hammer — Godspeed You! Black Emperor	2025-06-27	Godspeed You Black Emperor show at District Music Hall in Norwalk, CT	Purchase
127	Javascript	2025-08-07	Astoria Tech Meetup meeting	Gift (from Tea)
128	Gomez — FEZ	2025-08-09	Long Island Retro Gaming Expo at Cradle of Aviation Museum	Purchase

[*] The Collabora's Logo pin is not in the display since it's magnetic, but you can see it in the previous blog post.

The full spreadsheet wouldn't fit comfortably in this page, so the table above contains only the main columns. You can download the full spreadsheet from the link below which also includes the pin's authenticity, maker and additional comments:

Pins spreadsheet

Song of the Month: Soundgarden - Black Hole Sun

2025-08-31T00:00:00-03:00

Just two days ago I went to an immersive theatrical experience called Viola's Room. It started in a girl's bedroom with dim lighting and a few beds made on the floor as in a sleepover. On the headphones, a beautiful song started playing. The moment was so magical it's hard to describe. I felt like a kid again for a second. And I knew I'd always remember this moment in the future (dès vu). If you have the chance to go, I deeply recommend it.

Anyway, the song that was playing, as I later found out, is my song of the month for August:

Soundgarden - Black Hole Sun

Song of the Month: Junko Yagami - Bay City

2025-08-12T00:00:00-03:00

Today I want to try starting something new. Every month I'll share one song that stood out to me throughout the month.

For this first one, even though it's already midway through August, I still really feel the need to share this song that has marked my days throughout July and even June. Many of my strolls around the city on sunny days have been accompanied by it, making it the hymn for this Summer to me.

Without further ado, my song of the month for July is:

Junko Yagami - Bay City

A new theme for the blog (now with 0% JavaScript!)

2025-05-04T00:00:00-03:00

I've recently updated the blog's theme and I felt I should make a post not only to document the reason but also to archive pictures of the blog before and after the change for the future.

When I created this blog, I chose nikhil-theme as the theme because I found it minimalist and pretty. (Even prettier after I switched the fonts to be the same as the ones in the https://www.thirtythreeforty.net/ blog)

However, quite some time later as I was looking into the theme's template HTML files, I was surprised to discover that this very simple theme contained JavaScript. I tried loading the page with JavaScript removed and found that it was only required to get the hamburger menu to expand on mobile devices. That didn't sit well with me. A simple blog like this shouldn't require any JavaScript at all. And I found it even more annoying that it was only required because of one single feature.

At the time I had already been making other changes to the blog and was not looking forward to completely switching the theme, so this thought stayed in the back of my mind until recently, when I finally decided to do something about it.

In order to decide on the new theme for the blog, I started by looking through pelican-themes (the same place where I originally discovered nikhil-theme) but only found one or two themes that didn't use any JavaScript and they didn't really appeal to me.

Eventually I remembered about the seirdy.one blog. That's a blog I follow which has excellent write-ups, and there's a whole page about the design standards that the website adheres to. The standards focus on minimalism, compatibility and accessibility, which are all qualities that I would like for my blog, so I decided to adopt Seirdy's theme for my blog.

Looking through seirdy blog's source code I noticed it uses the Hugo static site generator whereas my blog uses pelican, so adapting this theme wouldn't be just a simple copy-paste.

Ideally I'd have gone through Seirdy's HTML templates and understood them all so I could adapt them to my HTML Jinja2 templates for pelican, but I honestly didn't have all that energy in me. So I went for the lazy solution: I copied over the CSS files and tweaked my templates until the website looked good enough. This means I'm probably not following many of Seirdy's standards, but at least the website is leaner, without any JS (and even external CSS), which was the main goal.

The resulting source code change can be seen in this commit (and also this and this for follow up fixes).

As for the resulting changes on the website itself, the pictures below show different pages before and after the theme change, respectively.

Home page:

A blog post:

Tags:

About:

I won't lie, I already feel nostalgic and a bit sad to see the old theme go, but I believe making the website more minimalist is worth it.

One thing lost in this transition was syntax highlighting for code snippets. I assume it's intentionally avoided by Seirdy due to accessibility concerns, but I should confirm that at some point and if it's not the case then consider adding it back.

Even though there are still a few details left to iron out, I'm quite happy with the change and particularly relieved there's no more JavaScript running when seeing this blog.

In search of the perfect pin necklace

2024-11-16T00:00:00-03:00

Last year, on the "Collecting pins" post, I mentioned that I usually wear my pins on a beanie, but was still looking for a good alternative for the warmer months.

After thinking more about it, I realized a necklace would be the way to go. It doesn't depend on any specific piece of clothing and it's pretty visible, not only to others but also to myself.

I started researching for pin necklaces and found a few different solutions for the pin-to-necklace converter:

The guide is great, but I don't feel rubber pin backs are safe enough. And neither the plastic nor the leather converters looked good in my opinion. But the locking pin back one seemed perfect! So I ordered one right away.

Actually, I ordered two. The reason is that some pins have two posts, side-by-side, so two pin backs would be needed to keep those pins correctly oriented on the necklace (see pictures at the end). By buying two necklaces I could transfer the converter from one of them over and have a necklace with two converters, allowing me to wear any pin in my collection.

Once the necklace arrived and I started using it, I quickly realized there was a big problem with it though: the pin kept getting turned around! That totally beats the purpose of the necklace. I concluded the problem was the chain, which needed to be switched for something stiff, like a leather cord. I settled on this braided leather necklace (the 20inch in length and 3mm in diameter variant): https://www.amazon.com/Flexible-Braided-Leather-Necklace-Pendants/dp/B095W7J7D7

Once that arrived, I took two pliers, two jump rings (since the original ones wouldn't close around this broader necklace) and used them to put the two pin back converters on the new necklace. And it worked perfectly, no more pins getting turned around! I also liked the way it looked much better, the chain was too shiny and thin for my taste.

On top of that, since the jump rings I used are much broader than the necklace, it's super easy to get the pin back converters in and out of it. That way, when I'm wearing a pin that only uses one back, I take the other out so it's not dangling and making noise.

Finally, here are the pictures of the end result:

One-post pin:

Two-posts pin:

Pins being worn:

The only issue is that the one-post pins, like the Dwarf Fortress one above, always rest tilted to one side. I don't think there's a way to avoid that on a necklace though. In any case, that's a minor issue and I am incredibly happy with the result. I've been wearing pins everyday since!

CardOS: Now compiling without Arduino!

2024-07-31T00:00:00-03:00

In the "CardOS: Writing an OS for the Cardputer" post I shared about the OS that I'm writing for the Cardputer and that the next step was to move away from the Arduino toolchain. It took me two months but I finally did it. The end product was this commit but I'd like to go over the process to get there in this post.

I started by running the Arduino commands to build and flash with verbose output enabled and saving that output for reference. This way I could see what exactly Arduino was doing and understand the steps needed to get from a source to a binary flashed to the chip's memory.

I then got to doing the obvious needed changes to convert from Arduino to C: I renamed the single source of the project from cardOS.ino to cardOS.c, changed the setup and loop functions into a main function, and changed the compilation step in my Makefile to rather than calling Arduino, call the ESP32 C cross-compiler (xtensa-esp32s3-elf-gcc), which had been installed on my machine by Arduino and whose path I learned from Arduino's output.

Next I ran the compilation, and fixed each of the errors that were thrown by the compiler. Namely I had to add a few includes, function prototypes, change the way a couple variables were defined and pass the -fno-builtin flag to the compiler so it would allow me to define my own stdlib functions. With that, I had an ELF file and needed to figure out how to do the flashing.

From looking through Arduino's output, I learned that esptool.py was the command used to convert the ELF file into a binary, and then called again to flash the binary into the chip. Besides the application code, there were other things being flashed: a bootloader and a partition table. In order to keep things simple, I did a quick test to verify whether I could ignore them for now (that is, assume they were already flashed before): I tweaked the OS code and used the Arduino toolchain to build and flash just that and the tweak showed up on the Cardputer, so the answer was yes.

With that in mind, I updated the Makefile to call esptool.py to convert the ELF generated by the compiler and then flash it to the board. At this point I had the whole procedure to get from the source to the flashed binary on the board figured out (or so I thought), so I ran it with make upload. But it did not work, the screen on the cardputer just wouldn't turn on.

I realized assuming all the code would just work in the new setup was too optimistic and decided to simplify the test as much as possible: I changed the code in main to simply drive the GPIO1 pin high and connected an LED to that pin. But even then, the LED didn't turn on.

This is where I got stuck for a while. The issue was clearly somewhere in the application binary. My two main theories were that either the binary itself was malformed, or I was missing some initialization code, since Arduino included a bunch of extra files during compilation.

Configuring the addresses in the ELF file

Hoping the issue was in the binary itself, since identifying missing initialization code from Arduino could take a while, I started investigating it.

I used readelf to see the contents of both my ELF and the one generated by Arduino and compared them. The biggest change in the header was this:

Mine:

Entry point address:               0x40017d

Arduino's:

Entry point address:               0x40376778

Arduino's ELF had much more code in it, so I expected its address to be higher, but this was too much of a difference. It looked like some base address was intentionally set to something different.

I looked up the meaning of the "Entry point address" in an ELF and confirmed that this is the address where execution starts from. So getting it wrong could definitely make my code not work at all.

Looking further through the ELF's content:

Mine:

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000000 0x00400000 0x00400000 0x0183c 0x0183c R E 0x1000
  LOAD           0x00183c 0x0040283c 0x0040283c 0x001a1 0x0075c RW  0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .text .rodata .eh_frame
   01     .ctors .dtors .data .bss

Arduino's:

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x001020 0x3c000020 0x3c000020 0x2a678 0x2a678 RW  0x1000
  LOAD           0x02c000 0x3fc88000 0x3fc88000 0x0c018 0x0d8a8 RW  0x1000
  LOAD           0x039000 0x40374000 0x40374000 0x0ca97 0x0ca98 RWE 0x1000
  LOAD           0x046020 0x42000020 0x42000020 0x1f347 0x1f347 R E 0x1000
  LOAD           0x000000 0x50000000 0x50000000 0x00000 0x00010 RW  0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .flash_rodata_dummy .flash.appdesc .flash.rodata
   01     .dram0.dummy .dram0.data .dram0.bss
   02     .iram0.vectors .iram0.text .iram0.data
   03     .flash.text
   04     .rtc_noinit

After reading up online about ELF program headers and sections this started making sense.

Looking at the program headers on the Arduino ELF, there is indeed an address very close to the entry point address: 0x40374000 on segment 2. That is the base address I was suspecting. The code that starts executing then must be in the iram0.text section, since sections with text in the name represent code, and that one is mapped to segment 2. The questions that come to mind are:

Why is this address used?
How do I set this address in my ELF file?

To answer the first question, I went looking in the ESP32-S3 manual. Figure 4-1 shows a diagram with how each memory range is mapped. The starting address of the segment used for the entry point code in Arduino, 0x40374000, is inside the range 0x40370000 - 0x403dffff which is shown as mapped to SRAM through an instruction bus which totally makes sense!

But reading on, Table 4-1 further breaks down the memory regions and names the region containing 0x40374000 as "Internal SRAM0". In its description, it is mentioned the first 16KB of the space can be reserved as a cache for instructions stored in the flash memory. If we do the math, that means the usable instruction memory starts at... 0x40374000, exactly the address that was used for the code segment in Arduino's ELF! So that explains where this address came from.

To sum up, the problem I uncovered is that the ELF file I was generating had the code assigned to addresses that do not map to a memory region that can be accessed by the ESP32's processor to fetch instructions (ie accessible through an instruction bus) and therefore it couldn't be executed.

What was left was to answer the second question: How do I set the right address in my ELF file?

This was a big gap in my knowledge, I had no idea. But looking through Arduino's output, I saw that the compiler was being passed some .ld files through a -T parameter. Inside one of them called memory.ld I found addresses being defined! The -T flag entry in the compiler's manual page revealed these files were linker scripts, so I knew what I needed to learn about next.

I found this wonderful blog post about linker scripts that taught me everything I needed to know. With that information I was able to write my linker script to assign the right addresses to each ELF section: Not only for the code (.text), but also for the zero-initialized variables (.bss) and other variables (.data), whose addresses I figured out similarly by referencing the manual and the Arduino ELF. A few additional sections also had to be assigned to get rid of linker errors.

As you might have noticed from the ELF contents, Arduino has two code sections, iram0.text and flash.text, while I only have one, .text. For that reason, I decided to assign my .text section to flash since it has much more space. However, there was still no sign of life from the cardputer. Since I noticed that my code was small enough to fit entirely inside the SRAM, and Arduino's code started from SRAM, I decided to try that, and it worked!!

The ESP-IDF's application startup flow documentation describes what the second-stage bootloader (which is the one I'm relying on from Arduino) does and does not, and it mentions that it is the application's duty to finish setting up the flash MMU. That is probably why using the flash address for the code didn't work and why Arduino splits the code between an SRAM and a flash part. So at some point I will need to set up access to flash, but for now SRAM was enough.

Initialization code

Now that I had at least an LED turning on, I changed the code back to full operation, that is, initializing the display, rendering the shell and reading the keyboard.

The screen started getting cleared as usual but stopped midway. I removed the screen clearing routine temporarily and the shell prompt was written to the screen. I could type in, but after a short interval my position would go back to the intial one. I realized that the Cardputer was getting reset after a precise interval.

I remembered from the application startup flow documentation that the bootloader enabled the watchdog. And since now the application code was fully provided by me, I had to take care of it myself: either keep feeding the watchdog or disable it.

I chose to disable the watchdog for simplicity (as always), and after referencing the watchdog section in the ESP32-S3 manual and adding a few register writes to the code, it was done: the system no longer crashed. This allowed the shell to stay on screen but it was garbled:

Again, remembering from the application startup flow page, the application code (me!) was responsible for initializing the .bss section to zero. Since the .bss section contains the zero-initialized variables, if I don't initialize it, those variables will contain trash, which is what I was seeing here.

I didn't know how to do this the proper way, but I could definitely do it the dirty way, that is, manually listing all global variables in a function and zeroing them out 🙈. Which I did and it worked! (Note: I have figured out how to do it properly since and I did it in this commit)

Finally I had a working OS without needing Arduino to build! And that's the full story behind the "Move from Arduino to generic C build flow" commit.

There was still one difference from before the move though, it was much slower. That's because, for the last time referring back to the application startup flow page, the application is supposed to set the CPU clock frequency to the desired value. The default frequency is quite slow, but the startup code implicitly embedded by Arduino would set it to a higher value. I fixed that in a follow up commit.

Further improvements

With the move away from Arduino done, I was finally able to work on some much needed improvements.

The first thing I did was to split the monolithic source file into several different files in this commit. I was really looking forward to this and it felt great to finally do it 😌. Now the code is much more organized, it's easier to find things and to focus on a single component.

I also enabled all the main compiler warnings in this commit, including warning about implicit fallthrough in case switches, which would have saved me some minutes investigating a bug early in this project.

Conclusion

This was another great step for the project. I had a lot of fun and learned so much from it.

The next big thing to tackle is implementing SDcard read and write, and a file system. It will likely be a while before I get that done and come back with an update on the blog. Feel free to check the repository for the latest updates in the meantime if you're curious!

CardOS: Writing an OS for the Cardputer

2024-05-25T00:00:00-03:00

I recently got the M5Stack's Cardputer. I was motivated to get it because I knew other people personally that had it too, so we would be able to share our progress, but I was worried it might end up being just another board that sits in my closet forever untouched. Still I decided to give it a shot.

Once I got it in my hands, it really won me. It has all that's required to be a computer: full keyboard for input, display for output, battery for portability, microSD card slot for persistent storage, plus WiFi and Bluetooth for connectivity. The fact that it's a very limited computer, and that it's so tiny, only makes it more charming to me.

I am really interested in the idea of implementing a full system from scratch in order to understand all the different layers and interactions. I've also always wanted to make my own operating system (OS). So before I knew it, I had set on a goal: I was going to make an OS from scratch for the Cardputer.

Now, when I say from scratch, I don't mean necessarily starting from an empty file, just that eventually all the code should have been written by me. One thing I remember from Andreas Kling's (creator of SerenityOS) videos is that he didn't start writing his OS from the bootloader, but from the user interface, and that allowed him to see his changes take effect right away and work in small steps towards the full OS. I also remember Tarn Adams (co-creator of Dwarf Fortress) giving the same tip about how to keep motivation when developing a game. And both of those are people I dearly admire, so I took their advice.

So with that in mind, I started my project from one of the Cardputer's demo code which already had a working display and keyboard, and I wrote a basic shell on top of that.

Only after that did I start implementing my own keyboard and display support from scratch and removing the code from the demo.

Getting the display to work

Implementing display support was much trickier than I expected and got me stuck for a while. By then I had already implemented my own keyboard support, tested it and it worked. But I couldn't even get the display to light up, even after carefully reading its datasheet many times and implementing the whole initialization procedure.

Eventually I took a step back and tested more basic functionality and realized I did not have output GPIOs figured out. This was really surprising as it was needed for the keyboard, which I had already tested. But what I hadn't realized was that even though my code was able to toggle output pins to scan the keyboard, it was implicitly relying on the demo's initialization code to have working pin output, which is why when I replaced all that code with my own display implementation neither the display nor the keyboard worked.

But as often times is the case for unexplainable behaviors, that wasn't the only issue. As it turns out, the ESP32 manual also showed the wrong address for one of the registers needed to configure the GPIOs. Luckily it also showed the right address in a different place, so I eventually noticed it.

So after I implemented support for enabling the output GPIOs, and corrected the register address, I was able to fix the keyboard support to be completely self-contained, and was finally, after a lot of sweat, able to implement display support and remove the dependency from the external library.

This has been the biggest achievement on the project so far, and I'm really proud of it!

Current status

This is where the project is right now:

{image}/cardos_demo.mp4

In other words, keyboard and display support are there without relying on third party libraries. The display communication is quite slow, so there's a lot of room for future improvement 🙂. There's a shell with just a couple commands, including help to show the commands, clear to clear the screen, read to read an arbitrary memory address and led to turn the built-in RGB LED to the desired color. It's possible to cycle through the shell's history to repeat a previous command. And once the terminal fills the screen it automatically clears it.

I've also spent some time fixing bugs. At this point it felt like the OS was finally in a usable state, so I've tagged it v0.1.

Next steps

The next thing I'm working on is to move away from the Arduino toolchain. The main reason is that there's some code that gets implicitly added to the binary when using it, and I want to keep removing dependencies from my code.

Besides that, Arduino basically enforces the usage of a single source file. The cardOS source is currently a single file that is 1900 lines long (with almost 600 being the struct for the font characters) and I'm starting to get lost in it, so it's time to split the code to multiple files.

Finally, Arduino uses C++, and since I'm coding in C, moving away would allow me to use a C compiler instead, which will improve the error messages.

After that I'm going to keep making cardOS more of an actual OS. I want it to feel like a real hackable computer, so what I'm aiming at is to be able to open a text editor, edit a shell initialization file, save, reboot and see the effects of that change. That already gives me enough work: I need to make a text editor, a shell script language, and implement SDcard reader and filesystem support.

Further in the future I'm really curious about implementing support for the WiFi and maybe be able to send and receive IRC messages, if that's feasible. I'm sure it'll be hard, but should be fun too 😃.

And that's what this is all about: fun! It's been two months since I started the project, and I can confidently say this is the project I've had the most fun ever. So as long as it stays fun I'll keep going with it. There's no rush to get anything done, I'm just enjoying the ride 🙂.

vCard + RSS as an alternative to social media

2024-03-22T00:00:00-03:00

Last year after talking for a while with someone during a conference they asked me for my LinkedIn to be able to connect with me, to which I answered I didn't have one.

It was many years ago when I decided to leave social media. I don't miss it. Instant messaging allows me to keep in touch with the people that really matter to me in a much more personal way.

Still, this interaction stayed in the back of my mind. It would indeed be nice to keep track of these connections too. The people you had an interesting conversation with for just a few minutes. To be able to check how they're doing every now and then, who they're working for and what they're hacking on.

Essentially I would like for everyone to have an online profile and feed, which I could easily check and subscribe to, and that this subscription data could be stored locally. Why should that be exclusive to social media?

It's not. In fact, the feed part has already been solved for quite a long time with RSS/Atom. So by just having the URL for someone's RSS/Atom feed, you can add it to your feed reader and easily follow the person's updates wherever they're published to.

The profile part, a place with the key points about a person's current status (name, photo, location, employer, etc), is the one that has historically been tied to social media. Personal websites do many times have an "about" page with such information, but I find the lack of structure makes it way less useful than a social media's profile.

However, there's actually a widely used open standard for encoding profiles as well: vCard. It's mainly known as the format used on email clients to store contacts, where they're even synchronized between server and clients using the CardDAV protocol. But vCards are also used for contacts in your phone: it's possible to import and export contacts on iOS and Android using vCards.

So that's when I realized, if there's a widely used open standard for feeds, and another for profiles, why not join the two? By embedding an RSS/Atom feed URL in a vCard, it becomes a single file that has all the information needed to identify a person and follow their updates. Since it's a single file, it's easy to share it with others as well as store the vCards for all your connections locally.

Now one interesting bit is that though the profile information on the vCard might get outdated, there is actually a SOURCE property which is meant to contain the URL where the latest version of the vCard can be retrieved. So by hosting your vCard in a public URL, say your personal website, it's possible for people that have already downloaded it to keep it up to date.

As for how the feed URL can be encoded in the vCard, after skimming over the latest version of the spec, I originally intended to have it as a plain URL property. But I wanted the main URL in the vCard to still be the person's website, so some mechanism would be needed to differentiate between the two URLs. At first I thought of using a MEDIATYPE parameter in the URL, but the spec explicitly said that for protocols such as HTTP the Content-type HTML header should be used instead. However, as it seems, despite RSS/Atom feeds having standardized media types (application/rss+xml and application/atom+xml respectively), in practice, text/xml is used instead. In the end, this means that in order to tell that an URL in the vCard is for an RSS/Atom feed, the URL would need to be fetched and read. To avoid that complication, I decided, reluctantly, to use a non-standard property instead, X-FEED, for the feed.

Trying it out

In order to try the idea out, I created this repository with a couple simple scripts. First there's a vcard-render script that parses vCards and renders them using a provided jinja template. The idea is that you can use it to render your vCard and your connections' vCards in your website for example, which is exactly what I use it for in this page I've added to the website (it can be found from the About page). If you're curious as to how I integrated the script into my website, see this commit.

The other script is vcard-to-opml which extracts the RSS feed from the vCards supplied and generates an OPML file with them, which can then be imported into a feed reader. The idea here is to run this script on the vCards of all the people you have downloaded (your connections) so you can easily follow their feeds. I use newsboat as my feed reader, so I would add the following command to a crontab to run it periodically (as soon as I get some real connections!):

newsboat -i <(awk 1 ~/ark/etc/dav/contacts/5DEB-65D21680-2F1-4AEA3480/* | ./vcard-to-opml.py)

Note: awk is used here instead of a simple cat because apparently those vCards don't have a newline at the end of the file. I'm not sure if it's my email provider's fault, where I store my vCards, or vdirsyncer's fault, which is the program I use to have a local copy of my vCards.

Conclusion

If you're interested in trying out using vCards and RSS to connect to people as well, check out the repository that contains those scripts. And feel free to reach out if you do so, you can find my email on... you guessed it, my vCard 😉.

Even if no one ever uses this, at least I now have a proper answer for the next person that asks for my LinkedIn. "Just scan this QR code and you'll see my profile" 🙂.

Collecting pins

2023-11-23T00:00:00-03:00

It started innocently about one year ago as I was looking for a gift for a friend. We played a lot of League of Legends and Ragnarok Online back in the day, so I wanted to give him something to remember that. Eventually I came across these pins:

They were exactly what I was looking for so I bought them. That was going to be it, but I am still personally a huge fan of Ragnarok Online, and those pins were so cute that I thought, I should buy them for me too!

But after that I couldn't stop looking for more pins. I was hooked! I kept browsing Etsy, searching for pins from games I like, and favoriting them so I could find them later.

I also started looking for pins at every merchandise store I came by, which I'd never given much attention previously. I now actually had something of interest to buy! So after a few weeks I got a small collection:

Around that time I was thinking about attending PAX east. When I found out the event attracted a big community of pin collectors and hosted pin trading events, from this video, I knew I had to go.

In order to be able to conveniently bring my pins to PAX, and to bring back the ones I would get there, I bought a PinFolio Classic.

On my first day at PAX I didn't hold back and bought every nice pin I saw:

After lucking out and winning two golden cassette pins in a row from a carnival wheel that was setup there, I went to the pin trading room to trade some pins and meet new people. It was fun to recognize some of the same people that showed up on the video.

With PAX over, this was my collection:

(the Deviruchi pin is missing since it was pinned to my beanie)

Now, many months later I have finally filled up a display case with my collection. Note that I have some more pins, but I plan to use those for trading in future events, so they're stored in the pinfolio. So without further ado, here is my collection, labeled:

Poring. From Ragnarok Online. Fan-made (Teaberryhouse). The one that started it all! Link
Poporing. See 1.
/gg Emote. Came with 4.
Deviruchi. From Ragnarok Online. Fan-made (FinniChang). Bought together with 1 and 2. Link
Eevee. From Pokemon. Fan-made (Jackalandhare). I was buying two pins for a friend that loves Pokemon and the third was cheaper, so I got one for myself as well. Link
Fairy bottle. From Zelda. Fan-made (FinniChang). Link
Van Gogh's The Starry Night. Got it while visiting the Museum of Modern Art in NYC, where they keep the real painting.
Prince & Katamari. From Katamari Damacy. I played Katamari Damacy Reroll last year and thought it was really cool! And who can resist a spinning pin? Bought at the Fangamer stand at PAX. Link
King. See 8. Link
Ravenclaw. From Harry Potter. I bought it at a Harry Potter store in the London airport, where I had a connection. I don't really care about Ravenclaw, but it was the best-looking pin there.
Junimos. From Stardew Valley. Bought at the Fangamer stand at PAX. Link
Came with 11.
Babaa (Apparently that's its name!). From Neopets. I played a lot of Neopets when I was younger, so when I saw these available at a Geekify stand at PAX, I just had to buy some.
Strawberry & Chocolate Paintbrush. See 13.
Wraith Paintbrush. See 13.
Fight Button. From Undertale. Big Undertale fan, as will be obvious from this list, so I just went crazy. Bought at Fangamer stand at PAX. Link
Mercy Button. Came with 16.
Annoying dog. See 16. Link
Wonderville Arcade machine. Bought while visiting Wonderville with a friend, which is really a unique and awesome place!
Psychokinetic Raz. From Psychonauts 2. I haven't played the game yet, just the first one, but I had just finished watching the PsychOdyssey documentary and was blown away so I had to buy something to remember it. Bought at Fangamer stand at PAX. Link
Phantom of the Opera. Bought it on the way out after watching the play on Broadway. It was great!
Canada Wilderness. Gifted from a friend. He bought it during our trip to Toronto (at the store on the base of the tower, I believe)
Loto. Gifted from a friend. It's an old one got from the second hand market, unlike all the others in this collection. I love how it has actual numbers for each tiny square.
Frisk. From Undertale. Bought at Fangamer stand at PAX. Link
Toriel. Came with 24.
Sans. Came with 24.
Flowey. Came with 24.
Papyrus. See 24. Link
Undyne. Came with 28.
Alphys. Came with 28.
Mettaton. Came with 28.
Mask. From Mr Robot. Bought it second-hand online. It's from a 2016 Loot Crate. I really wish it was smaller.
Software Freedom Conservancy's Logo. Gifted from a friend.
L's Logo. From Death Note. Bought at a geek store in Antwerp.
Dwarf. From Dwarf Fortress. One of the reasons I went to PAX was to meet the creators of the game in person (and I did!). Shortly after a Dwarf Fortress pin was announced and I was sure to buy one online. Link
PAX East VHS. Bought at PAX. People at PAX will mostly only trade Pinny Arcade pins, so I bought this as a part of a 4-pin set to kick start my collection and be able to trade at the event.
East 2023 Logo. Came with 36.
Diamond Ore Block. From Minecraft. Got this from trading at PAX. Minecraft was one of the most influential games in my life, so I was super happy!
ATLAS and P-Body. From Portal 2. Got this from trading at PAX. I love Portal, but this is my least favorite pin out of the set, so I hope to trade it for one of others next PAX.
Raz. From Psychonauts. I was given this one for free by someone at the trading room at PAX. She said she already had too many pins and since I was just starting out and liked this one, she was kind enough to give it to me 🙂.
? Block. From Super Mario Bros. Bought at PAX as part of a 4-pin set to start my Pinny Arcade collection and be able to trade. Link
Mario. Came with 41.
Fairly OddParents. I watched tons of hours of this cartoon, so was really happy to find a pin for it. To my knowledge the only official pin. Made by Zen Monkey Studios. Link
Paranaguá train. Gifted from a friend, who took that train. I really need to go there one day wearing this pin 😃.
Kerbal Space Program. Gifted from the same person as 40.
Green Flame. From Acquisitions Inc. It's a tabletop RPG campaign played live during PAX. I watched it during PAX and got this pin to remember the moment. Link
Mae. From Night in the Woods. I bought it at PAX before even playing the game. I just really loved the style. Possibly my favorite pin.
Los Andes. Gifted from a friend who visited that place, and he's really into rock climbing so it's very fitting.
Goose. From Kickstarter. During PAX the Kickstarter folks had geese scattered across the stands of each project funded by kickstarter. If you found all of them you would get a pin. It was incredibly hard but I did it.
PAX Gold Mixtape. During PAX there was a carnival wheel that you could pay Pinny Arcade pins to spin and get one of the mixtape pins. I was extremely lucky and managed to get two of the golden ones. I traded one and kept this one.
Portuguese tile. Bought during a trip to Portugal.
Collabora's logo. Got it during the company meetup. Unfortunately it's magnetic, so I'm too scared to wear it on the streets and lose it.
Ecuador. Gifted from a friend who visited that place, the same rock climber as 48, see a pattern? 😝

So far I've been keeping my pins in a display case on a side table. I've thought about having them on a board on the wall to make them more visible but I'm worried they might get damaged over time being constantly exposed like that.

However I don't like to just leave my pins locked up in a case either, since that seems like a waste, so I wear them. I've found what works best for me is to wear them on a beanie. (See first photo) As a result I can't wear them during Summer (I'm still looking for an alternative here). I always keep it to a single pin, since more than that looks too noisy to me, and switch it quite frequently and when possible to match the occasion.

Now that this display case is full I'll need to buy a new one (or maybe I'll try a board this time). I'll probably make another post when that one gets full too, though that might take a while. I do have many pins on my wishlist, but I'm trying not to go too crazy with it. It's about the quality not the quantity after all, and I don't want it to get to a point where I can't appreciate every single one.

Meeting the Linux community at my first ELC

2022-09-29T00:00:00-03:00

This month I attended the Embedded Linux Conference Europe, which was co-located with the Open Source Summit Europe, and took place in Dublin, Ireland. This was my first time attending a Linux conference in-person as a contributor and it was a very special experience.

This wasn't my first ever in-person Linux conference though. Back in 2018 I attended linuxdev-br. It took place right at Unicamp, where I was an undergraduate student. I remember watching Sergio Prado's presentation on how to write an LED driver and being fascinated with the fact that a physical LED could be turned on and off by simply writing to a file.

That conference was actually a major turning point in my life, as it was there that I first met Helen Koike, who invited me to attend the weekly meetings for LKCAMP, where they were teaching about the kernel inner workings and guiding people to submit their first patches and become contributors. And after learning a lot from them and having some contributions merged I eventually made my way into Collabora.

Fast-forward to 2022 and here I was in Dublin, for my second ever in-person Linux conference, this time at a much bigger one, and having worked for more than a year at Collabora, as an active contributor to the Linux kernel.

As a contributor, I'm used to interacting with the other kernel developers through the mailing lists on a daily basis. And while that works efficiently for the project, dealing with just text replies and email aliases can feel very distant.

And that's why meeting these people in real life was so reinvigorating. It's a very nice change of pace to be able to talk to them in a more expressive and human manner, not just about work but life in general. And to actually see with my own eyes that these are all still real people, with their own life, aspirations, food preferences. It reminded me that the whole world really is built by people just like me.

It was also my first time meeting with my colleagues at Collabora, which was absolutely amazing! Picture here. That's me all the way on the left 🙂.

There's a photo album of the event from the Linux Foundation here in case you want to check it out.

I barely got back from this trip and I already can't wait for the next opportunity to meet everyone again!

(The Convention Centre Dublin glowing bright at night, on the other side of the Liffey river)

Moving the blog to Codeberg

2022-08-29T00:00:00-03:00

As someone who cares about FOSS, I'm always happy to move to a FOSS alternative when one shows up, provided there aren't any big drawbacks.

Back when I was in University and starting to learn the ways of Git, I only knew about two Git hosting options: GitHub, the mainstream but proprietary, and GitLab, the less known but more open alternative. Between the two, GitLab was the obvious choice for my personal repositories, including this blog.

A few months ago I learned about Codeberg. Codeberg provides a hosted instance of Gitea, which is a Git forge that is entirely FOSS. On top of that, Codeberg is backed by a non-profit, which makes it clear that it is community-focused. As far as I'm concerned, it doesn't get better than this, so I was eager to move all my repositories to Codeberg.

Most of my repositories are really just archives: I'm the only one pushing code to them, and as long as the commit history is available, there's no other feature they require. So the migration was pretty straight-forward. The only exception is the blog repository.

Codeberg blog setup

The one big missing feature in Codeberg which I relied on in GitLab for my blog is the CI.

My current setup for the blog in GitLab was to have a repository with the CI configured so that whenever I pushed new changes, the CI would run pelican to generate the static pages for the website and serve them.

On Codeberg, since there's no CI, whatever files are present in the special pages repository are directly served. This approach required me to set up two repositories: one for the source files, the blog repository, and another for the generated output to be served, the pages repository.

Note: It's also possible to use a pages branch in the same repository to serve the files, but the separate repository approach seemed neater to me.

But of course I didn't want to manage the extra pages repository, since that would be annoying and quickly make me miss GitLab's CI. So I added a new make deploy target in the Makefile to automate the process of deploying the site to the pages repo. It can be seen in this commit.

What make deploy does is:

Do the steps previously done in the .gitlab-ci.yml to generate the blog output, namely:
- Generate the translation strings (through make trans_deploy)
- Generate the static site pages (through make publish)
Commit the output files and push them to the pages repository

Since the output files generated from pelican need to be commited and pushed to the pages repo, the local output directory inside the blog directory now needs to be a git repository. To acommodate for that, the OUTPUT_RETENTION variable needed updating so that pelican doesn't delete the .git folder every time it runs.

I subsequently improved the commit messages for the pages repo through this commit, by formatting them with both the commit hash and description, in the same format that is used for Fixes: tags in the Linux kernel. This allows me not only to correlate any commit in the pages repo to the corresponding commit in the blog repo, but also easily tell from a glance what the latest changes were about.

I also added a .domains file to the retention since it is required by Codeberg for using a custom domain.

Conclusion

At first the need to version output files with Git put me off, but the fact that it's in a separate repository and it's easy to correlate the commits to the source ones made it a non-issue for me.

I did notice a couple advantages on the pages repo approach over CI. First, I don't need to think about the image and package versions I'm using for the CI because there's no CI. As long as my local system is able to generate the blog, I'm good (and I already needed to have a working local setup to test changes before pushing anyway). Second, as soon as I make deploy, the new changes are live, while GitLab's CI would have taken a minute or so to update.

So it took some little changes to have a good blog setup on Codeberg but I'm happy with the move. Codeberg feels like a nice new home and I'll be sure to take good care of it 🙂.

Using emojis in matplotlib

2022-07-20T00:00:00-03:00

Last month, as I was writing the blog post with all the statistics for the blog's two year anniversary, the "Blog statistics after two years" post, I decided that I really wanted to have a plot with emojis. From the moment I thought of this I knew it couldn't simply work and I was in for some fun.

When I first tried generating my plot with emojis, this is how it came out:

These emojis just have the outline and some are even missing. I wanted matplotlib to use a proper emoji font, like NotoColorEmoji or twemoji, which are colorful and look nice. I had both installed on my system but they weren't being automatically picked up by matplotlib.

After a bit of searching I figured out how to explicitly add the font to matplotlib:

matplotlib.font_manager.fontManager.addfont("/usr/share/fonts/noto/NotoColorEmoji.ttf")

And I also added fontname="noto color emoji" to the matplotlib call that should draw using this font (in my case xticks()).

By forcing matplotlib to use the emoji font I wanted, I no longer got those outlined emojis, in fact I didn't get any image at all 😝, just this traceback:

 Traceback (most recent call last):
   File "/home/nfraprado/emoji.py", line 3, in <module>
     matplotlib.font_manager.fontManager.addfont('/usr/share/fonts/noto/NotoColorEmoji.ttf')
   File "/usr/lib/python3.10/site-packages/matplotlib/font_manager.py", line 1092, in addfont
     font = ft2font.FT2Font(path)
 RuntimeError: In FT2Font: Could not set the fontsize (error code 0x17)

With an error code even, inviting me to dig deeper 😆. Since I'm not familiar with the inner workings of fonts, the first thing was to figure out what FT2Font was. And it turned out to be matplotlib's wrapper for FreeType, the library that renders the fonts.

The traceback error message was coming from this line, which meant that FT_Set_Char_Size(), which is a freetype function, was failing with error code 0x17. A look into freetype's error code reference revealed that it meant "invalid pixel size".

As I tried to figure out what differed the emoji fonts from normal fonts, I did notice something that seemed to be fundamental: Running fc-scan on the emoji fonts revealed that they had a pixelsize property which wasn't present in normal fonts. Furthermore, running ftview (from the freetype2-demos package) on the emoji fonts with any size resulted in the emojis being drawn in the same size, while normal fonts were correctly scaled.

At this point I needed to actually dig into the code, so I cloned the source for matplotlib and freetype, compiled, and set the environment so I could use them: installed matplotlib in a virtual environment with pip install -e . and pointed the LD_LIBRARY_PATH environment variable to the directory containing the build output from freetype.

After enabling debug logs in freetype (with FT2_DEBUG=any:5) and adding some logs of my own, I noticed that the difference in code run for the emoji fonts was due to FT_HAS_FIXED_SIZES being true, whose meaning can be seen here.

To sum up the issue here (to my understanding): emoji fonts are made by embedding bitmaps, which have a specific size. This is indicated by FT_HAS_FIXED_SIZES being true, and the size of these bitmaps shows up as the pixelsize attribute. When matplotlib is going to draw the font, it specifies the size it wants to render the font in, but since the emoji font has a fixed size, the freetype code expects the size passed in to be the same as the font size. This is because you can have multiple versions of the bitmaps, with different sizes, inside the same font, so freetype would pick the one with the size that was asked for.

With that in mind, I came up with this patch:

diff --git a/src/base/ftobjs.c b/src/base/ftobjs.c
index f66273f3d3cb..a59a61119702 100644
--- a/src/base/ftobjs.c
+++ b/src/base/ftobjs.c
@@ -3074,6 +3074,7 @@
     FT_Int   i;
     FT_Long  w, h;
 
+    ignore_width = 1;
 
     if ( !FT_HAS_FIXED_SIZES( face ) )
       return FT_THROW( Invalid_Face_Handle );
@@ -3101,8 +3102,6 @@
       FT_Bitmap_Size*  bsize = face->available_sizes + i;
 
 
-      if ( h != FT_PIX_ROUND( bsize->y_ppem ) )
-        continue;
 
       if ( w == FT_PIX_ROUND( bsize->x_ppem ) || ignore_width )
       {

What this patch does is force freetype to return the first bitmap option inside the font, even if it doesn't have the size we asked for.

With that change done, I then got a different error, now in matplotlib:

Traceback (most recent call last):
  File "/home/nfraprado/emoji.py", line 3, in <module>
    matplotlib.font_manager.fontManager.addfont('/usr/share/fonts/noto/NotoColorEmoji.ttf')
  File "/home/nfraprado/matplotlib/lib/matplotlib/font_manager.py", line 1134, in addfont
    prop = ttfFontProperty(font)
  File "/home/nfraprado/matplotlib/lib/matplotlib/font_manager.py", line 545, in ttfFontProperty
    raise NotImplementedError("Non-scalable fonts are not supported")
NotImplementedError: Non-scalable fonts are not supported

So, only now is matplotlib complaining that the font isn't scalable, which should have been the error to begin with... Anyway, I was curious to see if this would all work in the end, so I just removed the check:

diff --git a/lib/matplotlib/font_manager.py b/lib/matplotlib/font_manager.py
index f57fc9c051b0..08d8c28752cb 100644
--- a/lib/matplotlib/font_manager.py
+++ b/lib/matplotlib/font_manager.py
@@ -541,8 +541,6 @@ def ttfFontProperty(font):
     #  Length value is an absolute font size, e.g., 12pt
     #  Percentage values are in 'em's.  Most robust specification.
 
-    if not font.scalable:
-        raise NotImplementedError("Non-scalable fonts are not supported")
     size = 'scalable'
 
     return FontEntry(font.fname, name, style, variant, weight, stretch, size)

And finally got some output:

The emojis are black and white, and not the right size, but it's a step forward. At this point, what I was seeing looked a similar to what I had seen in a blog post and the linked issue on Github while searching around. The issue there is about the TTC format, so not the same thing, but the workaround to use a different backend that has better support for emojis, mplcairo, did sound promising.

So I installed mplcairo with pip and set the script to use it with

matplotlib.use("module://mplcairo.gtk")

And this was the output:

So the emojis actually rendered colored and scaled correctly with this backend! The only caveat was that the text on the Y-axis was rotated, but a quick search revealed that this was already fixed on the main branch, so I reinstalled mplcairo from git, and everything was golden!

I did think this investigation was going to end up in some fix being sent to either the font, freetype, or matplotlib, but with my current understanding of the issue, I'm not entirely sure what would be the right fix here. I'd need to investigate this a bit further to know, but given that I've managed to get it working for my purposes, granted with some hacky patches, I don't see myself looking more into this for now.

That said, my understanding is that mplcairo does the font scaling on its own for fixed size (emoji) fonts, so maybe making mplcairo check the available font sizes and ask freetype for the one available that is closer to the size it'll draw with, could be a way to get rid of the freetype patch. As for the matplotlib patch, maybe fixed size fonts could be allowed when the mplcairo backend is being used, since it clearly knows how to handle them. But again, more investigation needed to conclude anything.

In any case, with the two patches shown here and the mplcairo backend, I was able to generate the plot with the most frequently used emojis at the end of the "Blog statistics after two years" post. The complete script that generated the plot can be seen here.

Finally, for reproducibility purposes, it's worth to say that all of this was tested with the packages in the following commits:

mplcairo: 74c27c3dbd54 ("Tweak path search in build-windows-wheel.")
matplotlib: 60ae76b6b5b0 ("Merge pull request #23243 from timhoffm/take-over-22839")
freetype: e7482ff4c2a3 ("* src/lzw/ftzopen.c (ft_lzwstate_stack_grow): Cosmetic macro change.")

Blog statistics after two years

2022-06-20T00:00:00-03:00

Wow, it's been two years since I've started this blog! I'm really happy that I've still managed to keep up the one post per month. Hopefully I'll be able to keep it up for the third one!

For this post I thought it would be fun to look through some statistics on what I've done on the blog so far. To make this possible I've created a 'stats' plugin for pelican that generates the statistics I'm interested in. I then wrote python scripts using matplotlib to plot the collected data.

Plots & Stats

I've managed to keep up with the monthly posting schedule, but I haven't been able to always post on the 20th of each month as I intended, and it shows here:

(Open the image in a new tab to see it better)

In the beginning I hadn't yet decided on a date to post each month, but starting with 2020's September post I settled on the 20th of each month and I managed to more or less keep that up until May of the following year. After that I've almost never again posted on the 20th, and there's even some kind of upwards slope as the posts have been sliding to the end of the months.

This other plot shows the total accumulated count on each day of the month:

The 20th is still the day of the month I've posted the most on, with 8 on total, but there are more posts evenly distributed in the days after it, totalling 17! That's to say only about one third of my posts were published on the 20th like I intended.

This doesn't bother me much though. It would've been nice to keep the posts consistently going up on the same day, but posting every single month, whatever the date, is much more important to me, and I'm very happy I've been successful on that.

Changing subjects a bit, I always wanted to know the word count of the posts, so here it is:

There's no clear tendency, which is what I expected. The smallest ever post was the "Setting up mbsync to work with XOAUTH2" post with 553 words in English and 567 words in Portuguese, while the largest one was the "Managing my tasks using VIT" post with a whopping 2727 words in English and 2747 words in Portuguese. The second largest post is the "Owning my Kindle" post not too far below, while all the rest are below the 2000 words mark.

This graph also allows for comparing the word count between English and Portuguese, and it's clear that the Portuguese version is almost always slightly bigger. This agrees with my perception, as multiple times I've increased the sentence's length as I try to express the same thing in Portuguese which in English I was able to express with a common technical term.

In total, all English posts sum up to 32718 words, while all Portuguese ones sum up to 33488 words. So my Portuguese posts are in average 2% larger, which is honestly less than I expected.

It is worth mentioning that this word count is done at the final generation stage of the article, when it is already in its HTML form, meaning that all the content that shows when it's published is there, with the difference that the HTML tags that could add to the count are stripped. In practice, this means that the total word count is the post's text plus the code blocks. So one important element that doesn't show up in this plot are the images, which can be seen here:

Not all of the posts have images, but some of them have quite a few! In fact, the "Owning my Kindle" post, which was the second largest post, is the one with the most images, 16 in total!

Now, this part is very arbitrary, but I wanted to account for the number of images in a post when comparing the amount of content in each one. They say a picture is worth a thousand words, but for my purposes that seems a bit excessive. Instead counting each image as 75 words, a normal paragraph, seems reasonable. I also decided to make GIFs be worth three times as much, 225 words, since their dynamic nature makes them much richer. With this in mind, I plotted the total content estimate of all posts:

Again, the numbers are totally subjective, and this still doesn't account for the audio files I've composed and linked to in the "Learning music theory by writing melodies" post nor for the code that I wrote on some repository and just linked in the post, like in the "Keeping track of my packages" post. But given its limitations, I'd say this is much closer to my perception of how much content each post has.

A notable diference is that on this new plot, the "Owning my Kindle" post became the post with the most content.

Anyway, going back to more objective metrics, I plotted the number of links to other posts, or cross-references if you will, each post had:

So most posts don't link to others (to be expected), and the ones that do only have a couple links at most. That said, this very post would be much higher than the rest on that plot, but I'd say it is an exception, since it's an introspective post about the blog.

What about any links?

Every post has at least one link, which is not surprising in the least, this is the web after all!

And finally, the most important question, how many emojis??

Not as common as links (thankfully?), but not bad either. I do seem to be getting more consistent in my emojis, whatever that means.

But how good is it to know how many emojis are being used if we can't see which ones?

Of course the most common is the smiley face, hard not to smile when you're talking about something that you're interested in. On second place we have the tongue face for the funny moments. And a sprinkle of other random emojis.

This plot was way harder to do than it seems, but more about it next month (probably).

Conclusion

Even though it can be difficult to put myself to write about something every month, I really do enjoy the result. This monthly ritual turned out to be therapeutic to me, as I get the chance to build a new brick in this safe space that is my blog, so I feel like I'm advancing in my life even if nothing else happens.

The need for a post every month also encourages me to actually keep doing things that interest me, otherwise I'll end up without anything to write about!

Discovering the comfort of loudspeakers

2022-05-27T00:00:00-03:00

In 2014, while on a school trip to Germany, I bought a Razer Tiamat headset. The sound was great and for the most part it was comfortable. Its only issue was the clamping force, which was a bit too much, and gave me headaches after long hours of use.

I made great use of this headset and it served me all the way up to 2021. By that point, the leather on the earpads was mostly gone, and the jack connector had already broken twice and needed resoldering a new one. One of the times the contact on one of the wires wasn't very good, which caused one side to be louder than the other and needed compensating in software, but then the compensation needed changed with changes in volume, which drove me crazy and I remember being incredibly happy when I finally fixed it.

So by 2021 a new headset was already overdue. Since the clamping force was my main pain point (quite literally), I was going to try to find one that had the lowest possible while still being reasonably priced.

Searching for a comfortable headset

Searching online I found a site called Rtings that reviewed and ranked headphones, with one of the criteria being the clamping force. I filtered their list by this criterion, and settled on the HyperX Cloud Alpha since it had one of the lowest clamping force and wasn't very expensive.

But when it arrived, I was very disappointed. The clamping force was much higher than what I had on the Tiamat and I easily got headaches after just a couple hours using it. I tried flexing it overnight, but that didn't seem to make a difference. So I eventually just started avoiding to use the headset, and used the builtin notebook speakers instead, which sound awful, but they at least aren't painful.

A few months later I had moved to the US and had a much wider and accessible array of options at my disposal, so I decided to try it again. I wasn't going to get it wrong a second time, so I made sure to try out as many headphones as I could at physical stores, B&H being the biggest one I knew.

To my surprise however, almost all of them had a big enough clamping force to be uncomfortable for me. Even the Bose QuietComfort 35, which from my research was one of the most comfortable headphones available, was a bit too tight for me. I did try some that were actually okay, like Sony Mdr7506, Phillips HP9500 and Koss UR20. And funnily enough they were also cheaper than most others. But these headphones seemed like they were so loose they were about to fall off. There didn't seem to be a sweet spot for the clamping force.

At this point, I started thinking that maybe my problem was inherent to headphones. I had already thought of earbuds, but the issue with these is that they deform inside your ear, causing a different kind of pain. Then I suddenly thought about loudspeakers. Could they be my answer?

Searching for a good speaker

As I researched I found a very well researched and tested Speakers ranking from Wirecutter. Looks like this is one of the most well known series of rankings, but I hadn't come across it before so I was happy to find it.

I came very close to ordering one of their top picks, but I kept feeling like a subwoofer would be more important to me than the flat response they were prioritizing, since my use case is almost exclusively listening to music, and not producing.

They did have as their "Also great" the Klipsch ProMedia 2.1 THX, which has a subwoofer. As I researched around, I noticed that while other rankings had different top picks, this speaker was also consistently appearing on other ranks. And it seems like this speaker has been around for a long time, so if it's still relevant in these rankings, then it must be really good.

The downsides for the Klipsch usually mentioned the need for extra space for its bulky subwoofer and lack of conectivity eg. bluetooth - the audio input is exclusively through a 3.5mm jack cable. I didn't have any problem with the extra space needed: The subwoofer goes on the floor, and I have plenty of free space there! As for the jack input only: I was going to keep it constantly connected to the Line Out port on my docking station, so no worries here either.

Finally, to get a better feeling for how important a subwoofer would be for me, I opened some songs that I liked and that had a deep bass in them in Audacity and used the Filter Curve effect to try and imitate the frequency response graph that the Wirecutter article showed for the speakers with no subwoofer to try and see how it would sound like. This proved very useful and it was clear to me that without the lower frequencies, these songs lost a lot of their substance. So I concluded that I really wanted a subwoofer, and the Klipsch seemed like the best speaker with one, so I went for it.

The Klipsch ProMedia 2.1 THX

And oh boy... This has to be one of my happiest purchases in life. First of all, because of the change from headphone to loudspeaker. I can't believe I can finally listen to music all day long without feeling anything at all on my head. This is magic.

Second, the subwoofer really brings the music to life. Not every song makes use of it, but more songs than I thought. You can mainly hear it (and feel it!) from the bass and the percussion in the songs. There is a subwoofer dial and I keep turning it down and up again to compare the difference it makes on the sound and it's impressive.

As for the downsides, there's a bright power LED between the dials and no ON/OFF switch, which means this LED is always on. But a small piece of tape to cover it easily solves the issue.

Another thing is that by having such a good speaker fixed to my desk, I do miss it when I want to take my notebook to a different room. If I could somehow bring it with me in a portable way, that would be wonderful... But then we're back at the concept of headphones and we already know where that path leads 😝. I also miss a bit the sound isolation (in both ways) that you get from headphones, but that's way less important than comfort to me.

Conclusion

I find it interesting that when I was a kid, the computer in my house had a pair of speakers as was usual. But since then I've only had, and seen other people use, headphones, so I subconsciously considered it the only option. Realizing that's not the case, even though loudspeakers do seem a lot less common nowadays, and going back to them now feels poetic in a way.

I do still have my hated headset and I use it for calls, since the echo from the speakers would be very bad. But as soon as the call is over I'm really happy to get the headset off and get back to the comfort of my speakers 😌.

Learning from SerenityOS

2022-04-29T00:00:00-03:00

One day I was scrolling through Reddit as usual, when I saw a post linking to this blog post: I quit my job to focus on SerenityOS full time. I was intrigued by the backstory, the premise of this OS, and also by the fact that its development was being recorded on Youtube.

Now, almost one year later, I have watched almost every video Andreas has posted in his channel since and a few of the older ones too. His videos are so good, they're entertaining, inspiring and I've learned so much from them. And he's such a nice human being too. The fact that such a positive project came out of his own therapy is just perfect. I think he definitely found his calling in life and I'm really happy for him.

SerenityOS is such an interesting project itself. There's code for the kernel, libraries, services and applications, all there in a single repository. It invites you to become familiar with the whole system and to see how each component interacts with the others and fits with the whole. Furthermore there's such a huge variety on what kind of problems each component is trying to solve that it feels like there's always something interesting to explore on Serenity.

But having everything together isn't only helpful for learning the code base. When you make the whole system, it's possible to do some really cool integrations. Some of my favorites features on Serenity are:

All data exposed by the kernel through procfs-like files is in JSON!
The Profile Viewer app is not only able to show system-wide profiles with tree-view stack traces but also even signposts set by traced applications during interesting events!
There's a nice markup language to design the GUI for the applications, GML, and the GML Playground app shows you a live preview of the GUI as you write!
On every application (unless it opted-out) it's possible to open a command pallete with all commands on that app!

Another advantage that comes from making the entire stack is that changing APIs is much easier, since they control both sides. That allows faster iteration and for better ways to do things to evolve over time without needing to maintain older iterations for compatibility concerns. That said, that flexibility doesn't extend to the ABI and interactions with the user. And although so far it has been the case that these haven been freely changed as well, I wonder how much of this is due to SerenityOS still being seen as Work-In-Progress and there not being daily users yet. But perhaps it being "a system by us, for us", the ability to iterate and improve upon even the ABI will be seen as more important than keeping it stable. I guess we'll see.

When it comes to contributing to the project myself, so far I've done just a couple ones, mainly the most basic support for showing album covers in the sound player and adding proper support for multiple keyboard layouts in the KeyboardSettings app. But I had a lot of fun working on them, and look forward to contributing more.

Edit: on the next day from publishing this post my contribution on Sound Player was showcased in SerenityOS's April Update video! 😃

One thing that really got me thinking is Serenity's goal of having 1990's-like GUI. When I started to use and learn about Linux some years ago, I was immediately attracted by CLI and TUI programs and have preferred them since. They represented a very different paradigm from the GUI programs that I was used to in the Windows world. On the terminal there was no fear of exposing all the functionalities in a very flexible manner, there was much more room for customizations and performance tended to be better.

But after seeing Serenity's approach, I think my disregard for GUI applications has been a bit misguided. With the right mindset, GUI programs can be quite flexible, customizable and performant. And on top of that they can look consistent and be aesthetically pleasing. What else do you want from software? In short, I'm actually confident now that I'd prefer using GUIs for most tasks, it's just that the ones out there don't tend to focus on the right things for me.

SerenityOS still does feel a bit like a playground. And it is fun to play around with. But I also agree with a lot of its goals, and it is really rapidly growing, so it is a very promising OS to me. On one hand, I wish it was ready for everyday use so I could switch to it already, but on the other, that would take all the fun of getting there (both when contributing myself and also seeing Andreas' videos!). Plus it's easier to contribute to it if there are still a lot of rough edges 🙂.

Running LineageOS on my Nexus 5X

2022-03-30T00:00:00-03:00

This month marks one year since I bought a Nexus 5X and started using LineageOS, so I thought I'd share my experience.

Setting up

About one year ago, a friend told me he found a really good deal for a Nexus 5X online. The Nexus 5X is an old phone by now, launched in 2015, so it tends to be fairly cheap, even though it's still a rather decent phone by today's standards. The offer he found was for a new Nexus 5X, which came inside the original box, perfectly conserved, and still cost only R$ 300. Really good deal.

At this point I had already been using Linux on my computer for some years, but my phone, arguably the machine that has access to the biggest amount of personal data, was still running Android, which is an OS that is not entirely FOSS, and that bothered me quite a bit.

I had already had the desire to switch to LineageOS, the FOSS alternative to Android, for some time now, but since it was my only phone it felt like putting too much at stake. With the possibility of getting a new Nexus 5X suddenly I saw the opportunity of having a tried and tested (and loved) phone with LineageOS, and if all went wrong, my older phone would still be there with everything the same way it was. So I decided to get a Nexus for me as well.

I followed my friend's Nexus 5X guide and had some personal help from him in order to understand the required software components to be able to run a LineageOS installation completely de-Googled. In my case the phone was brand new so I could use the image without the BLOD patch and enjoy full Nexus 5X performance.

The installation itself was easy to follow (granted I had a guide and a mentor). And at the end I had the phone with LineageOS, MicroG and Magisk, completely de-Googled and yet able to install software that relied on Google Services when needed.

Issues

Overall the experience is great, but there are two main issues:

Frequent camera crashes: basically every time I open the camera, it crashes on the first try, and I need to reopen it and then it works. Annoying, but not enough yet for me to go fix it.
Flaky GPS support: GPS was the hardest thing to get working. I remember finding some pointers in this post, and finally got it working by simply enabling the "Mozilla Location Service" and "WiFi Location Service" location modules, but it didn't always work and would have an initial delay before it started working. And then I moved countries and can't get it working at all anymore...

But those are software issues and could be eventually fixed. There is however one single hardware limitation on the Nexus 5X that made me disappointed as soon as I realized: There's only 16GB of internal storage and no external storage card slot.

For the apps themselves 16GB can actually be ok, but for taking pictures or videos and storing music, it wasn't enough.

Searching for more space

But I didn't give up immediately on the matter. The phone was so close to perfect, I needed to find a workaround for the lack of storage space.

The Nexus 5X has an USB-C OTG port, so I realized I could buy a USB-C flash drive, the smallest possible so that it wouldn't be too intrusive, in order to effectively expand the storage space.

After researching for a bit, the most compact USB-C flash drive I found was the Kingston DTDUO3C/128GB. So I went ahead and bought it. Right after it arrived I noticed that it comes with a little keyring and that my phone case had a slot for that, so I tied it there and thought it looked cute. It's also way easier to keep it always together with the phone this way. That said there are some drawbacks: the USB drive sometimes gets stuck in my pocket, and also hits the screen. This is how it looks:

However, as I inserted the flash drive I was disappointed to see that my music player didn't recognize any of the music in there. It seemed that the flash drive was just like a dumb storage and I could only move files to/from it from the file manager.

But given that this was weird, I thought there needed to be a way around it. After all, it's just an arbitrary limitation imposed by Android. After a bit of research, sure enough, a post in the XDA forum came to rescue with a one-liner solution:

su
sm set-force-adoptable true

And just like that, reinserting the USB flash drive made the music player app hurringly start indexing all the music files it had just found in the flash drive. It worked!

There is one big caveat with this however, which is that I need to remember to always insert the USB drive before opening the music player, and closing it before removing the drive. Otherwise the player no longer sees any music on the device, and the next time I plug the drive in, it spends a couple minutes indexing all the files again (and crashing a few times during the process).

Given that I'd need my music files (and possibly photos) to be synchronized with the computer, I was also interested in getting Syncthing working with folders in the USB drive.

To my surprise it didn't work at first: Syncthing has read-only permission to external storage. But I quickly found out that giving it root permission removes this limitation, which can be done through Magisk. Not the ideal solution, but at least this gives a working setup for music!

I was also curious to see if I could have the camera save photos and videos directly to the USB drive. This would lift most of my concerns about being short on storage.

In order to not fiddle with the main camera app settings, I downloaded the Simple Camera app to try this out on, and configured it to save to a folder in the USB storage. To my surprise it simply asked for permission to have write-access to the USB, not requiring to be run as root like Syncthing did.

I then tried recording a video on it. On the first try it threw an error and asked me to select the image quality and I lowered it to 1080p. I think the write speed on the USB couldn't cope with whatever the original resolution was.

After that though, it did manage to successfully take pictures and record videos. However right after I stop recording a video the app closes, but the file does seem to be flushed correctly to the USB storage and sync'ed with my computer.

But actually so far I have avoided using the USB storage for the camera (by using the original camera app), because, unlike for music, taking pictures/videos is time-sensitive, and I don't want to risk having my pictures fail to save or take multiple tries (the app crashing is enough already). In any case it's good to know that it works, to some extent, for when I want to use the camera and the internal storage is full.

USB as internal storage

But something else happened when I set that force-adoptable configuration. While browsing in the settings, it looked like I now could actually format the flash drive as internal storage. This would mean that the flash drive memory would be used transparently, even to hold app instalations. But a question came to mind: "What would then happen when I removed the flash drive?". There was only one way to find out if this was usable or not...

It took something like 30 minutes for the flash drive to be formatted and most of the internal storage data to be moved to it. But at the end of that I had successfully extended my storage by 128GB. And it felt... really sluggish. The whole system would stutter all the time. Turns out that the USB flash drive speed is a lot slower than the internal storage. And now that the system depended on those transfers it was really noticeable.

But maybe I could live with a slower phone in exchange for more storage space, right? What about the big question: was does happen when I remove the USB drive? This is really important to know given that I'd need to remove it at least to charge the phone every day or so. And the answer is: the phone doesn't like it all. The system reboots 😝.

I spent almost three days like this: With a phone with a giant 128GB storage space, but that was laggy and would reboot every time I needed to recharge it (which also happened more often as it turns out that the flash drive drains quite a bit of battery while plugged in).

At the end of those three days I had had enough of it so I reformated the flash drive as external storage and went back to the previous setup. It was an interesting (although frustrating) experiment.

App recommendations

App-wise, I try as much as possible to use apps from the F-Droid store, but there are a handful of apps that I still need to use from the PlayStore. Luckily, there's the builtin app AuroraStore for that, which at least provides a privacy-friendly version of the PlayStore.

For the apps that I have installed from F-Droid, here are the ones that I really like:

AntennaPod: Podcast player.
DiskUsage: Storage space usage viewer.
EtchDroid: USB image creator. I've covered it in the "Resurrecting a computer on the go" post.
Infinity: Reddit viewer.
KeePassDX: Password manager.
Mullvad: VPN client.
NewPipe: YouTube viewer.
OsmAnd~: Offline map viewer based on OpenStreetMap.
Pluvia: Weather forecast viewer.
Puzzles: A collection of good puzzle games.
Syncthing: File synchronizer.
Termux: Terminal.
Vinyl: Music Player.

Conclusion

I don't know how long I'll go on using this phone, but I'm surely not going back to Android. I'll either stay with LineageOS or maybe take the plunge to PostmarketOS, if the user experience is in the JustWorksTM level by then (or not if I feel like I suddenly have a bunch of free time 😝). Either way I'm going to make sure my next phone has a bit more storage space 🙂.

Learning x86-64 assembly basics

2022-02-25T00:00:00-03:00

Recently I decided to learn assembly. I already had a reasonable understanding of how it worked due to some classes that touched the subject in university, however I never had the opportunity to really write assembly code.

Since my everyday computer is an x86-64 machine, it made most sense to learn assembly for this architecture, so I could avoid the need for a VM. I started with only the desire to get my hands dirty with assembly code, and not any particular objective or project.

At first I was alternating between trying things out and researching on the web just to understand enough to get a bare minimum assembly file and commands that would assemble it and run. Eventually I stumbled upon the book that would guide me: x86-64 Assembly Language Programming with Ubuntu.

This book is free, recent and had the perfect scope for me: it's aimed at people that already have a good grasp of programming, but are new to x86-64 assembly, and it shows some theory and concepts, but there are plenty of exercises to learn from practice.

It was pretty fun to work through that book, and it worked well for me to create some familiarity with x86-64 assembly. I'm sure there are still a bunch of things to learn on the subject, since the book only gives a basis, but it was enough to teach me some interesting things.

Signedness and two's complement

The biggest lesson to me was a better understanding of signedness. I'm used to seeing int and unsigned int in C, and to watch out for using the wrong signedness, but it wasn't as clear to me how that worked at the assembly level.

The first thing to have in mind, is that the type concept present in higher level languages like C (like if a number is signed or not) is completely absent in assembly. The computer memory stores only 0s and 1s, and it's up to you, the programmer, to interpret what they mean: is 01011000 the number 88, the character X, the POP AX instruction? With only that single byte, you can't even be sure of the size: maybe those are really 8 boolean flags in a single byte, or part of a 4-byte signed number. Without context it's impossible to tell.

If the same representation can mean both a signed or unsigned number, depending on the context, that means that when operating on those numbers, you as the programmer have to use the right variant of the instruction to give that context to the computer.

While going through the book, the following arithmetic instructions were presented for unsigned numbers:

add adds two numbers
sub subtracts two numbers
mul multiplies two numbers
div divides two numbers

And the following instructions were shown for comparison between unsigned numbers:

ja compares two numbers and jumps if the first one is above the second
jb compares two numbers and jumps if the first one is below the second

And sure enough, shortly after, the signed variants of those instructions were also shown:

imul is mul's signed variant
idiv is div's signed variant
jg is ja's signed variant
jl is jb's signed variant

But wait, what about iadd and isub? That's the thing, the way x86-64 represents negative numbers is through the use of the two's complement system, which has the useful property of allowing addition and subtraction to be done in the exactly same manner for both signed and unsigned values.

This means that there's only one way to add, independently of the signedness, and it's using add. There's no iadd. Likewise for subtraction.

So the interesting conclusion is that for addition and subtraction it doesn't matter if you use unsigned int or int for the variables in C. The unsigned keyword is there for you to tell the compiler to use the right variant of the instruction in the generated assembly, which is required when you're comparing numbers (ja vs jg, jb vs jl), multiplying (mul vs imul) or dividing (div vs idiv). But thanks to two's complement, in addition and subtraction there's no way to get it wrong 🙂.

Side note: interestingly, while writing this post, I read on the Wikipedia page that two's complement also works the same for multiplication, but only if you do a sign extend of the two operands beforehand. Which makes me think that if the mul instruction always did the sign extend step, no imul instruction would be required as well, but that would probably increase complexity (and cost) in the logic circuitry.

Other interesting lessons

The other thing that interested me the most was to realize that local variables are nothing more than adding more space to the stack. And that this is done simply by subtracting the stack register rsp by the total number of bytes needed for the variables at the start of a subroutine.

Also interesting was to learn how there are calling conventions to standardize on:

which registers are used to pass arguments to subroutines and in which order;
which registers can be overwritten by a subroutine and which should be left unchanged. When using the latter, its current value should first be pushed on the stack so that it can be restored before returning.

And what about the magic main() function that the C compiler expects in every C program? Assembly doesn't need compiling, so no need for that, but turns out a different magic label is expected by the linker: _start.

Some other things that were interesting to do in assembly:

Making syscalls
Exploiting a stack buffer overflow
Interacting assembly code with C code, and vice versa.

Lack of a good GUI

One thing I missed was a good GUI application when debugging the assembly programs. It would have been really helpful to have one that showed the values of expressions in tooltips when hovering, that was able to follow labels when clicking, and so on.

The book recommends using DDD, which is a GUI, but it felt clunky and really outdated. I went for using GDB together with the peda plugin, and that worked reasonably well, but being a CLI, every inspection required divining the correct command, so it took more time to get oriented.

Conclusion

This was a great experience and I hope to get back to it and further my knowledge past the "basic" level for x86-64 sometime in the future. Seeing what's happening at the assembly level really helps better understand the higher level languages, and value the way they hide complexities below!

I've uploaded the code I wrote for all the book's exercises to this repository. I don't expect it to be useful to anyone since it's simple stuff, but it's there either way.

The only exercise that I couldn't actually finish was the last one. There's very little information on the book about how to do it, and during research of the topic online I eventually got demotivated and started learning about other subjects instead. Maybe one day I'll give it another try. If you do know how to do it, get in touch! 🙂

And even though I couldn't finish that last exercise, while researching about it I ended up learning about how to use the asm syntax for GCC through this guide, to embed assembly in a C file, and also about the Compiler Explorer which seems a great way to learn about assembly and C by just seeing what assembly is generated from a given C code, so I'm calling this a win!

Making internal linking in pelican effortless

2022-01-20T00:00:00-03:00

One interesting feature of Pelican, the static site generator I use for this blog, is the internal link expansion syntax with {}. It is documented here. Some examples are {filename}, {static} and {author}. The purpose of the syntax is to have shorter and easier aliases to link to internal content in the blog. For example, {filename} can be used to link to other files, like posts.

The idea is good, but to my needs it fell short. What I really need internal link expansions for in my blog are images, posts and code.

Images

The idea for images is really simple. I write my posts in rst, and this is how an image is included in this format:

.. image:: path/to/image.jpg

The way I structure the files in the blog (which you can see in its repository), is that inside the content folder where all content is, the sources for the posts can be found inside articles/<year>/, and the images can be found at images/<post_id>/, where <post_id> is a string that identifies the post in which the image appears (it is the trans_id property of the post, derived by its filename and used to associate translations of the same post, but I'm calling it post_id here since it makes more sense in this context).

From this structure, if I use a relative path from a post to one of its images, it'd have to be something like ../../images/<post_id>/image_name.jpg. The {static} expansion can be used here to simplify it a bit: {static}/images/<post_id>/image_name.jpg. We can do better than this though 🙂.

It'd be way better if this path could be really shortened. All images are inside the images/ folder, so that should be implied. Heck, while we're at it, might as well make the <post_id>/ part be derived from the current post's id. That would make it perfect, since the only information left is the image name, which is the unique thing about the link.

The first step in implementing this custom logic was adding the linker plugin to pelican. It allows you to implement your own {} link expansions through python classes.

And the second step was implementing a {image} expansion through this commit. This is the commit where I updated all image links to use {image}. Feel the joy! It's so much neater 🙂.

Posts

Linking to posts is a little trickier, but not too bad. The idea is, I somewhat frequently want to link to another post I wrote previously on the blog. Using the {filename} expansion, it's not too bad:

I've shown that `in a previous post`__.

.. __: {filename}08-task-context-en.rst

But it could certainly be better. The {filename} expansion is relative to the current file, so if I'm referencing a post from the same year, it's like in the example above, but referencing one from a different year requires an additional ../<year>/ in the path. Also, I shouldn't need to write the whole filename. Ideally I'd only need to type what's unique to the post, that is, its post_id. Yes, even the language suffix (-en or -br) can be omitted, since I can derive it from the current post's language.

So far so good, it could be done with a bit more logic on top of what was done for {image}. But since I'm already improving things, I'd also like to take this chance to better standardize the text I use in the links. Sure, using a text like "in a previous post" blends well with the surrounding text, but it isn't immediately obvious that the link is to another post in my blog.

So the idea is to have a link expansion that not only maps to the right path to the post, but also automatically sets its text to have the post title. One final touch is that when setting the text I want it to also take into account the language of the post: if it's in English, the text should be in the format '"Title of the post" post' and if it's in Portuguese, 'artigo "Title of the post"'.

Updating the link's text is not something the linker plugin can do natively, so I first needed to extend it to enable that in this commit. With the basic mechanism in place, I actually implemented the new {article} expansion in this commit.

All that work pays off, as now I can show you how it looks by pointing to the very first post I wrote on this blog through a simple {article}tasklist: "Creating movie and game lists using Taskwarrior" post

Of course I also changed all references to blog posts to this new amazing expansion in this commit.

Code

This is where it gets messy... You see, it's quite common for my blog posts, being technical, to have code blocks amidst the text. The way I include the code is by using the include rst directive and giving it the path to a separate file containing the code.

The issue is, unlike the image rst directive and rst links whose target appears in the final HTML, which makes it simple to edit them inside pelican, the include directive and its path are processed by the rst reader at an earlier stage. This means that the normal link tweaking methods in pelican can't be used here (like the linker plugin).

I could put the code inline instead of using the include directive and avoid this problem altogether, but when the code is more than a few lines, I feel like it would pollute the post source file too much to have it inline.

Well, if you've read the "Blog customizations" post you might remember that I already have a custom rst reader. So in order to implement a link expansion for code, I had to extend this reader to substitute any {code} occurrence inside an include directive with the code file's path. That's what I did in this commit, by copying the code from pelican's rst reader and making a few changes.

It's worth saying that this kind of customization I'm doing (and was already doing) by overriding the rst reader is not really stable. If there are changes in pelican's rst reader, I might need to either stop updating pelican or reimplement their changes in my own custom reader. But since there isn't really a less intrusive way of implementing this, I'm willing to take the risk.

Despite the ugly changes, in the end it's all worth it when you look at the improvements in the post sources. This is the commit where I updated them to use the new {code} expansion. I dare you to say it wasn't worth it (please don't say it).

Conclusion

And here we are. Took some work, but after these changes, internal linking on my blog is effortless! Anything that makes writing blog posts easier is well worth it to me.

The menus in my system

2021-12-28T00:00:00-03:00

Another post going through stuff I set up for my desktop environment a couple years ago and that have used ever since 🙂. This time I'll show the menus I've created using rofi.

So, what is rofi? It's basically a program where you feed a list of options to it, and it shows a window where the user can filter and choose an option. So simply put it is a simple universal selection menu.

Rofi also has some default menus, like one to launch applications and another to switch windows, which are probably its most common use. I also use these menus, but since they aren't something I customized myself, I won't talk about them.

My menus

The menus I created with rofi are: power, screenshot, unicode and music.

Power

This is probably the most important menu to have when your desktop environment doesn't have its own power menu (e.g. in my case, I use sway which is just a window manager and doesn't have any menu of its own). This menu is bound to Super+Shift+P.

Each option is probably self-explanatory. They're sorted from least "damaging" on top to most "damaging" on the bottom, so that I don't accidentally shutdown and lose my work when I just want to lock the screen, for example, since there's no confirmation pop-up.

Since lock is the first option, simply opening the menu and pressing Enter already locks the screen. The other option I use the most is hibernate, which I just need to press h for it to get selected, followed by Enter to hibernate. In the rare cases I need to really shutdown I use w, and for reboot, I use re. Of course I can always cycle through the options using Ctrl+N and Ctrl+P as well, but the letters are usually faster.

The reason there's a lot of space between each icon and the text is because each icon has a different width, so in order to have the text of all entries aligned, I had to put a tab character after each icon. The same is done in the other menus.

Screenshot

The screenshot menu is bound to Super+Shift+S.

The screen option takes a screenshot of the whole screen (including external monitors). It calls grim under the hood to take the screenshots.

region invokes a cursor that allows me to select the rectangular shape that will be screenshot. This is done by calling slurp, which gets the selection from the user, and feeding the resulting coordinates to grim through its -g flag.

window also invokes a cursor, but it is used to select a single window to screenshot. This is done by parsing the dimensions of the windows from sway, and then feeding those as the options to slurp which will then only allow one of those rectangles to be selected. The command to do that, available in slurp's README, is:

swaymsg -t get_tree | jq -r '.. | select(.pid? and .visible?) | .rect | \"\(.x),\(.y) \(.width)x\(.height)\"' | slurp

colorpicker invokes a cursor as well, but doesn't take any screenshot, instead the pixel that is clicked will have its RGB value copied to the clipboard.

Unicode

The unicode menu is bound to Super+Shift+U.

The purpose of this menu is to make it easier to access characters that I don't normally have mapped in my keyboard. It shows each character followed by its name. Since there are so many characters, I use two columns on rofi so I can see more at once.

To use this menu I write the name of the character I want and, after it gets selected, I press Enter to copy the character to the clipboard. I can then paste it wherever I need it.

One example usage is to get the ordinal characters used in Portuguese: ª and º (called Feminine and Masculine Ordinal Indicator, respectively). These are mapped in the Portuguese keyboard layout, but to me it's easier to search for them in the menu when I want them than finding them in the keyboard.

This menu is generated from python using the unicodedata.name() function to get the Unicode name for each character, and chr() to get the actual character. wl-copy is used to copy the character to the clipboard.

Music

The music menu is bound to Super+Shift+M and it's my favorite! Each option opens its own sub-menu. It has all that I need to always be able to quickly play the music I want.

One nice detail is that if a song is already playing, whatever you choose will only play after the current song finishes. Otherwise, if no song is currently playing, it starts playing immediately. This way the current song is never stopped midway which is great.

Also, differently from the other menus, the music sub-menus allow (when it makes sense) selecting multiple options by using Shift+Enter instead of just plain Enter when selecting, although I seldom use this.

Let's see each one of the sub-menus.

Playlist

The playlist menu allows me to start playing any of my playlists. Randomization is automatically turned on when I select a playlist, so that the order of the songs in the playlist aren't always the same.

The playlist I listen to the most, Saved, is the first one on purpose. This way when I just want to start listening to something, it's as easy as Super+Shift+M, Enter, Enter. Listening to background music when I have to concentrate is also pretty easy: Super+Shift+M, Enter, Ctrl+N, Enter.

In case you're curious about how the playlists themselves are created, I've already talked about that on the "Playlist generation with MPD" post.

Song

The song menu is for when I want to listen to a particular song. Each entry shows the song name, artist and album. rofi doesn't support showing multiple strings in the same entry like this by itself, so to achieve this I need to construct a string with the same width as the rofi window, with each field occupying an equal amount.

To have a more or less predictable width I fix rofi's width to be character-based by putting width: -100; in its config. I then make this value available inside the python script for this menu by simply reading the value from this config (not pretty, but works). I then calculate the width each field should have and add spacing and/or truncate each of the fields individually as needed for them to keep the right width.

Since the font I use in rofi isn't monospaced, the width I set in the config is just an estimation and not precise in any way. So there can be a bit of extra slack, but it works well enough.

Queue

The queue menu shows the current queue of songs that are playing. I use it whenever I want to see which songs will play next or play a different song from the current queue.

When this menu is opened the cursor starts on the currently playing song.

Album

The album menu lets me choose an album to play. One important distinction is that playing an album turns randomization off, so that the songs in the album are played in the right order, while the Playlist and Artist menus play with randomization turned on.

Each entry shows the album and artist name. On the front there's also a % to mark what I call "full albums". The criterion is that if I like at least 80% of the songs in an album, I mark it with a %, and listen to it whole, even the songs I didn't quite like. On the other hand when I like less than 80% of the songs in an album, I just listen to the ones I liked.

The idea behind the "full album" concept is that I think there's some value to listening to an album as a whole, but I also don't think it makes sense to listen to a whole album when you don't like most of it. So I have that 20% slack so that albums don't need to be perfect before I'm willing to listen to them wholly.

Artist

The artist menu is the one I use to play all musics and albums from a single artist, in no particular order.

Current

The current menu is a very new addition! I noticed that I'm sometimes listening to some playlist and a song plays that makes me want to listen to its whole album or to all songs from that artist. This menu allows me to do exactly that.

Options

The options menu gives access to some extra functionalities. update causes MPD to update the music database, which is sometimes useful. random toggles the music randomization. I rarely use this since I have already configured my menus to turn randomization on or off depending if it's an album or not. update playlists re-runs my playlists generation script so they get updated.

Other considerations

Those are my menus and what I wanted to share in this post, but it's worth mentioning a few other points.

First, there's another rofi menu that use, although it isn't one that I created myself. It's called rofimoji and allows you to very easily search for and copy emojis.

Second, it's worth mentioning that rofi officially only works on X11, so I actually use the rofi wayland fork, as I've mentioned on the "Moving to Wayland" post.

And finally, in order to more easily create rofi menus from my python scripts, I've wrapped rofi in a python function with the following prototype:

def select(prompt, options, multi=False, args=[]):

where prompt is the string that shows in the prompt, options is the list of options that can be chosen from, multi is whether multiple options can be selected or not, and args allows passing additional arguments to rofi. The return of the function is the option(s) that was/were selected (just the option if a single one, and a list of the options if more than one). If the selection was aborted an exception is raised. I considered using python-rofi instead of creating my own, but that module returns the index of the selected entry instead of the entry itself, which I found would just make it more convoluted to use. Since it was pretty easy to create my own module, I just went for it.

That's it. I really like how easy it is to make menus using rofi from python 🙂.

The blocks in my status bar

2021-11-26T00:00:00-03:00

Five years ago when I moved to the i3 window manager, I started using its status bar, the i3bar. It is text based, and it's up to you what gets shown there. However it is not very modular: it's weird to combine different information to be shown since everything has to be joined in a single string manually.

About one year later I discovered a program to solve that issue: i3blocks. The way it works is that in its config you define the blocks that you want and what script will be run for each one. The text output for each script is what will be shown for that block in the status bar.

My blocks

I currently have 9 blocks in my status bar: time, schedule, task, battery, keyboard, storage-root, storage-home, update and music.

Time

Probably the most obvious block. I can't think of a status bar that doesn't show the current time...

This block shows the date (weekday, day, month and year) and time. Having the time showing in bold is a detail I really like, but don't recall where I got the idea from.

Schedule

This block shows me the current schedule i.e. what I should be doing right now. As you can see I should be reading a book instead of writing this blog post right now... But I have to hurry if I want to post this! 😝

I have shown this block previously in the "Organization beyond Taskwarrior" post.

Task

This block shows my current task context and the number of tasks I have currently in my inbox, the number of stuck projects, and the number of tasks with near due dates.

When all three are zero, this block is hidden, although that is very rare. As you can see, I haven't had enough time to sort the tasks in my inbox lately 😅.

I also showed this block already in the "Organization beyond Taskwarrior" post.

Battery

This block shows the battery charge percentage. The icon reflects the current charge (out of 4 possibilities), and when the battery is low the background becomes red to catch my attention.

When the battery is charging and at a high percentage, this block is hidden.

Keyboard

This block shows me the current keyboard layout. Since I only use two layouts, pt-br (ABNT2) and en-us, and most of the time I use the english one, this block only shows in red when I'm in the portuguese layout. When I'm in the english one, it stays hidden.

I introduced this block after too many times trying to use vim and having forgotten that I was in the portuguese layout.

Storage

These two blocks show the available and total storage spaces on my disks. The first (computer icon) shows for the root partition, while the second one (home icon) shows for the home partition. These blocks are almost always hidden, unless the free space gets low, in which case they show up with the red background.

Note: The free space shown in the pictures wouldn't normally be considered low, I just forced them to appear here.

Update

This block shows the number of packages that can be updated. It only shows if the number is high enough, otherwise the block stays hidden.

I used to have this block turn the background red if the kernel package needed to be updated, since I noticed that in Arch Linux updating the kernel doesn't keep the modules for the previous version around. This means that updating the kernel should be followed by a system reboot, otherwise some weird bugs could happen due to the lack of required modules (e.g. USBs not working). However after I found out about the kernel-modules-hook package that solves this problem, I removed this check and update the kernel without worrying.

Music

This block shows me the current playing music. It shows the artist followed by the track name, and the icon reflects the current state: playing or paused. When no music is playing the block stays hidden.

Since the artist or track name can get very long, when they get longer than a fixed length, I omit the rest using "...".

Old block: cardapio-unicamp

This is not a block that I currently use, but it was my favorite one so I want to mention it. Its purpose was to inform me of the meal that was being served at the university's restaurant, so I could decide if I wanted to eat there or somewhere else.

There were two iterations of it. In the beginning, it simply showed the meal description for the next meal. After some time I changed it to instead show the quality of the meals for the whole week. I did this by matching the meal description with words I considered either good or bad. The end result was ten squares, each day separated by a | and with the first square for lunch and the second for dinner. The square colors showed the quality: white for normal, green for good and red (or magenta in this old color scheme) for bad.

The meal descriptions were queried using a short python program of my own, called cardapio-unicamp. The script to get the meal quality for the whole week is also available in that repository. The conversion from the |, + and - to the colored squares I don't have published, but it was a couple of simple (yet ugly) lines of bash, so I'm sure you can do something better if you want to have that 🙂.

Improvements

I really like my current status bar setup but it took time and improvements to get here.

For instance I find the consistency is key for each block to be easily identifiable: each block has its own color above and some empty space around it, and its icon is always the first thing on the left followed by the text.

I also take advantage of colors to reduce the amount of text needed on the blocks. For example, the task block has three different numbers, but no label for each, since each color already gives away their meaning.

Reducing the amount of stuff in the status bar helped me focus in what is really relevant at the moment. To achieve that I started questioning myself which information really was important. I don't really care about which WiFi network I'm connected to most of the time, and I used to have a block for that (and it's a very common one), so I just ditched it. And there's information that is only relevant sometimes, which is why I now have my blocks stay hidden unless their current information is relevant.

I also reduced the number of useless updates of the blocks by changing them to update based on a signal instead of a timeout whenever possible. So for example the keyboard block only updates when I press the binding to toggle the keyboard, which sends a signal to this block.

Hooking each block to a signal also makes the blocks update faster to asynchronous events. For example, I update the packages in my computer using an alias, which in addition to updating through pacman, sends a signal to the update block at the end. So as soon as I have updated the packages, that block disappears, and not after some timeout.

Over time I've also written some code to make the block definition easier. As with other parts of my system, I started my blocks in bash, but eventually migrated them to python. As part of this I wrote a print_i3blocks function in python that receives the icon, text and color for the block, and outputs it in the JSON format required by i3blocks.

I also have a update_i3blocks command that receives the name of the block and sends the appropriate signal, so I can update specific blocks from other commands in the system.

The one annoying bug

When I migrated the block scripts to python I started having issues with the blocks disappearing randomly. It took me almost a year to really dive into the i3blocks code and find and fix this issue. Although it was hard to find, it was also interesting to understand the issue. It didn't have anything to do with python, it's just that since python code runs slower than bash, it became more apparent.

Unfortunately, since the i3blocks repository is no longer actively maintained, if you're facing this same issue and want to fix it, you'll need to apply the patch and compile it yourself (though that is is pretty simple). Nevertheless I can't overstate how happy I am to finally get rid of this annoying bug. Now I can just relax and enjoy my status bar showing all, and only, information that is relevant to me 🙂.

Resurrecting a computer on the go

2021-10-23T00:00:00-03:00

Earlier this month I was spending a few weeks in another country. It was late at night and I was once again looking over my personal files and thinking if there was a better way to organize them in folders.

After thinking for a bit I decided on a new structure and moved some files around. In the meantime, I realized I hadn't updated my packages for some time, so I started an update.

At the end of the update I saw an error. /etc/mkinitcpio.conf was not found. My /etc/mkinitcpio.conf is a symlink to a file in my home as this makes it easier to track the changes I do in that config as part of my backups. And between the files I had just moved was precisely that config, so I updated the symlink to point to the new location. I ran the update once again and pacman reported "nothing to be done". I naively believed that everything was fine now.

So I rebooted to make sure that everything was working with the files in the new place. As I selected my system in GRUB, it reported that /initramfs-linux.img was missing. Same for the fallback initramfs. I was officially locked out of my system.

It was now clear to me what had happened. During the update the kernel was updated, and as part of that the initramfs was re-generated, but since the configuration file was invalid it was aborted. When I re-ran the update command, pacman only concluded that nothing had to be updated and said so. What I should have done is to manually run the installation command again for the kernel package to have the new initramfs generated. Since I didn't, there were no initramfs files left on my boot partition, so I couldn't boot.

I took a deep breath and started thinking what I could do. I only needed an USB flash drive with an ArchLinux ISO to boot from and re-run the kernel package installation to fix that mess. Since I was in a different country though, my equipment was limited. Thankfully I did have an USB flash drive, but no other computer to flash the ISO from.

Unless... Unless I could use my beloved Nexus 5X phone to flash it. Its charging port is an USB type C, and I suspected it worked as OTG as well. So I had the USB flash drive and a USB A to type C dongle, I just had to discover if it was possible to flash an ISO from the phone.

I immediately opened F-Droid and searched for an ISO flasher. To my happiness I discovered EtchDroid. Its description really resonated with me: "You can use it to make a bootable GNU/Linux USB drive when your laptop is dead and you're in the middle of nowhere.". Seeing it might actually be possible, I went to the Arch Linux page and downloaded the ISO.

I then installed EtchDroid and opened it. I selected the option to flash an ISO, selected the ISO file, plugged the USB flash drive to the phone and selected it in the app. I really liked the simple UI:

I tapped the button to start flashing and a notification popped up showing the progress:

And after a few seconds it was done:

I then unplugged the USB from the phone and plugged it to my laptop. I was really happy to see that the USB image worked and booted from it.

To fix the issue I set up my mountpoints, chrooted into my system, and did a pacman -S linux to reinstall the kernel package and have the initramfs regenerated.

With the initramfs back in place, I rebooted and everything was working fine again 🙂.

I'm really happy that despite my dumb mistake, I was able to solve it really quickly even in a limited environment thanks to this nifty app called EtchDroid. Definitely keeping this one around in case of future emergencies.

My journey to a good backup system

2021-09-27T00:00:00-03:00

I'm a bit of a data hoarder. I still have some of the first programs I've ever written, photos I've taken on trips and drawings I did many years ago, to name a few. And since I don't trust some company to store all of this data for me, it was pretty clear that I needed to do my own backups.

One lesson I learned early on reading online is that backups should be as easy to do as possible, preferably completely autonomous, otherwise you end up not doing them at all.

In this post I want to recall my previous backup setup and then talk about the current one. Hopefully it will be clear by the end that the current one is much simpler and better.

Ye Olden Days

I think it was in 2018 that I started thinking seriously about making backups. For that I needed both hardware and software. The hardware to house the backups, and the software to make them happen.

Hardware

I had a desktop computer that no longer worked due to a problem in the motherboard, but everything else worked fine so it was perfect to use for my backup system. The hard drive could be used to store the backups, the power supply to power everything, and the computer case would house all the components.

I started by removing the malfunctioning motherboard and noticed that the power supply didn't give out any voltage by default. Searching online for its datasheet I discovered which wires needed to be shorted for it turn on. Since the original power button on the case was a push button that connected to the motherboard, which in turn shorted these wires, I also removed this power button and put a switch button in its place. I soldered the switch to the power supply wires and fastened it to the case with some tape and thin metal plates I had lying around to give it some rigidity. This is what it looks like:

As for the main component, the computer that would run the backups, I had a Raspberry Pi that wasn't being used so it was the obvious choice. I got a micro USB connector from some old cable, opened the cable up, and soldered the wires to the 5V and ground wires from the power supply, that way I could power everything from the power supply, since the hard drive was already powered by it.

Finally, since the hard drive was an internal one, which uses a SATA connection to transfer data, I bought a cheap SATA to USB converter cable and used it to connect the hard drive to the Raspberry Pi. This is the whole setup:

(This is a current picture, it didn't have the Ethernet cable at the time)

That was it for the hardware setup, next was the software solution to make the backups happen.

Software

As I was thinking about which software to use for my backups, I stumbled upon this article. It presented to me the idea of incremental backups (only storing the differences in files from the previous backup), and backup rotation (only storing the N latest backups and deleting the older ones) which I found genius. It also relied mostly on rsync for the copies, which I was already familiar with. In the post they write a shell script to do the backup and retention, but I saw a comment mentioning that there was a tool that did exactly that, called rsnapshot. So I went for rsync + rsnapshot as my backup solution.

Very soon I faced some issues. The biggest one was that rsnapshot could only backup to a local storage. This meant that it needed to run on the Raspberry Pi, which was connected to the hard drive where the backups would be stored, and backup the files from my laptop.

Since the raspberry needed to connect to my computer, the normal thing to do would be to set a fixed IP for my laptop, but since I carried it with me to other networks, I thought it would be best to keep it with a dynamic IP. So I ended up with an ugly solution where my computer had the cronjob for the backup, in which it connected to the raspberry, updated its IP there and ran rsnapshot there. rsnapshot then used that updated IP to connect back to my computer and make the backup.

To make the data safe, the disk was encrypted using dm-crypt. This means that before rsnapshot began the backup, the disk was unlocked using the passphrase, which was saved as a file in the raspberry's system (very unsafe, I know!). After the backup it was re-locked and powered off.

One other issue is that rsync doesn't deal well with file/directory renaming or moving. It thinks that the previous one was deleted and a new one created. This was a big problem since I was always trying to find the best hierarchy for my files, which meant renaming or moving around the base directories and then suddenly all files had to be copied again...

That issue was further aggravated by the fact that the transfer between my computer and the raspberry was done wirelessly, so rather slow. To workaround this I added some logic to the backup scripts so that if the total size of the transfer was too big, it would just fail and notify me. I would then look at the diff, remember that I had done some renaming on the directories and copy them over manually to the backup through a temporary Ethernet cable.

Lastly, since the power supply was very noisy and I also didn't want to keep it constantly on, I'd have a daily alarm on my phone to remind myself to flip the little switch to turn it on just before the daily backup cronjob. And I'd turn it off later before going to sleep.

Nowadays

Some time has passed since then, and given that that setup worked but it was far from good, I've done some more research on the backup tools out there to see if there was anything better. Everything changed when I found borg.

Borg, or BorgBackup, is a program to make backups. It can backup locally or remotely. The backups can (and should!) be encrypted. And deduplicated! I'm not sure if renaming/moving files makes the backup take longer, but at least it won't take more space since all files are chunked and deduplicated independently of their path. Oh, and there's great documentation and an active community!

On top of that, there's another project called borgmatic. Borgmatic provides a single configuration file to configure borg's usage, like the repositories to use, retention policy and files to ignore for backups. So basically while borg provides the commands to do backups, borgmatic allows you to configure it all in a single file and it takes care of everything else.

In the end, borg + borgmatic make it incredibly easy to have secure and reliable backups. I now have a single cron entry to do my daily backups: 0 21 * * * cd ~/ && borgmatic --no-color --verbosity 1 --files >> ~/ark/etc/bkplogs. It's important to note that borg always uses relative paths in backups, so that's why I cd to where the folders I want to backup are first (my home directory).

Also since all the backup configuration is done in a single configuration file with borgmatic, it's easy to backup that file as well. So if I ever lose my files and need to recover from a backup, I'll immediately also have the backup setup ready to go again.

So yeah, borg is pretty much great and a whole lot better than any other solutions I tried before. There's a single catch though, if you're doing a backup to a remote location, borg also needs to be running there. So in these cases you have borg running on the local machine, doing the chunking and encrypting (so the remote already receives it encrypted and it's safe!), and on the remote machine another borg instance will be listening and storing the chunks in the archive.

Given that restriction, you can't backup to a remote that is a simple storage on the cloud. There are, however, very good alternatives focused on hosting borg backups, like BorgBase.

Now, I still use my raspberry setup, although I've now installed borg there instead of rsnapshot, and removed all my weird scripts. It's now just an additional entry in my borgmatic config. Also instead of having the disk password lying in the raspberry system I had the idea to send it as part of the ssh command, that way I can store it securely in my own encrypted machine. Oh, and I now rely on a remote server for daily backups and only do local backups once a week, that way I don't need to have the trouble of flipping that little switch every day 😉.

It's worth mentioning that recovery of backups is also very easy with borg. There are two commands (offered by both borg and borgmatic): extract and mount. extract recovers the file(s) from a given archive. Note that it will be saved in the same path as used in the archive, potentially overwriting your local files, so better run it inside a temporary folder first. mount mounts the archive on a local folder as if it were a filesystem, allowing you to browse the files.

Conclusion

As you can see, I've significantly improved my backup setup. And that's because it is less interesting, not the opposite. As much as I have fond memories of having my own system, which relied on multiple scripts to solve the issues with what I had, it was a hassle to maintain. It's great to have everything just work out-of-the-box with borg and borgmatic: I can even forget I'm actually doing backups now, which is refreshing.

Owning my Kindle

2021-08-29T00:00:00-03:00

Several years ago my mother gave me a Kindle Paperwhite 2. I have read a few books on it since, but it never felt like it was really mine. Locking the screen showed some annoying ad, all books bought from Amazon were protected by DRM, books transferred through USB only worked if they were in Amazon's .mobi format, and there was always the unease of knowing that every page I flipped could be being reported back to their headquarters.

Recently, I decided to put a stop to it and find out how to jailbreak my Kindle. I was willing to either make it mine or lose it in the process.

Jailbreaking

A little bit of search gave me a very well maintained forum post on how to jailbreak the kindle. In fact that mobileread.com forum seems to be the focal point on e-book reader devices jailbreaking and customizations. It's wonderful how there are these bubbles on the internet filled with specific information which you never find out until you search for it.

I started following along the guide, which is actually very simple: the jailbreak itself consists of plugging the Kindle through the USB to the computer, dropping a file there and getting it to run. At this point however, I stumbled upon a big problem which I had already suspected: the USB in the Kindle no longer worked.

The micro USB port on the Kindle should serve both for power delivery to recharge the device as well as data transmission, to be able to mount it on the computer as a data storage and send files. But somehow the data transmission no longer worked, so I couldn't send files over to the Kindle. At least power still worked, so I could still recharge it.

With the main data channel to the Kindle borked I realized my alternatives were either to find out the issue with USB, which I thought would be a USB controller hardware issue, and require swapping the IC with a new one, or checking if the Kindle had a working UART interface I could use instead of the USB. The latter seemed way easier if it was possible.

Jailbreaking through the UART

Indeed it was possible to access the UART in the Kindle PW2 as this post shows. It also shows how to open the Kindle and gives some ideas on making the setup permanent.

The only hard part in opening was the first step: getting the front cover to unglue from the rest. After stopping to play the guitar a few years ago, I never thought this was the way a guitar pick would come to use again 😝:

With the help of the pick I got the cover unglued and after unscrewing a few screws I got to the insides (the second photo shows the PCB screws also unscrewed and flat cables disconnected since I was looking around, but there's nothing of interest below the PCB):

At this point I was ready to test if the UART really worked. One issue was that the Kindle's UART works at 1.8V while my UART cable is 3.3V. To workaround this I used a simple voltage divider to lower the voltage for the Kindle's RX, and two bipolar transistors and a voltage regulator to increase the voltage of the Kindle's TX (I have to admit that I searched the web for this circuit, but at least after seeing it I was able to verify how it worked and will remember it for future projects 🙂). This was the temporary setup:

With this I ran picocom on the serial port and rebooted the Kindle. This is what I got (trimmed down to not be too long):

Terminal ready
Battery voltage: 4072 mV

Hit any key to stop autoboot:  0 
## Booting kernel from Legacy Image at 80800000 ...
   Image Name:   Linux-3.0.35-lab126
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3054728 Bytes =  2.9 MB
   Load Address: 80008000
   Entry Point:  80008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

[    0.097278] boot: C def:bcut:batterycut=1
[    0.274436] LPDDR2 MfgId: 0x1 [Samsung]

[...]

3.0.35-lab126 #2 PREEMPT Wed Sep 25 00:44:40 UTC 2019 armv7l
Press [ENTER] for recovery menu...       0 /BOOTING DEFAULT.
IP-Config: no devices to configure
kinit: Mounted root (ext3 filesystem) readonly.

[...]

info milestone:7.91:sy99:sy99
info system:done:time=7940:time=7940
crond[849]: crond (busybox 1.28.3) started, log level 8

init.exe: sshd main process (848) terminated with status 127
init.exe: sshd ain process endeinit.exe: recevnt pre-start proc


Welcome to Kindle!

kindle login: info milestone:8.24:sc01:sc01
info system_fs_loopbacks:mountingfs:Mounting compressed directories filesystem images:Mounting compressed directories filesystem images
info milestone:8.27:sc02:sc02
info milestone:8.31:/usr/share/X11/xkb:/usr/share/X11/xkb

[...]

info clickstreamHeartbeatMetricsFramework:Skipping metrics recording, as it is the same day::
info monitor:writing_file:file=/var/run/upstart/stored.restarts:file=/var/run/upstart/stored.restarts
Retrieved 383 keys for system/daemon/pmond/

Well, hello Linux 3.0, how long have you been there? 🙂

And my input worked too so I was ready to keep going with the jailbreak. I kept following the jailbreak through serial post until I got root, but to my surprise after that it shows how to transfer the jailbreak file by using the USB storage, which I don't have working. But since I just got the serial working, I didn't see why I couldn't use it to transfer a file, even though I had never done that.

Sure enough I found a way, and it's by using the xmodem, ymodem or zmodem file transfer protocols. All of them are provided under the lrzsz package on Arch Linux. Since the Kindle only had the rx binary, which is the receiver for xmodem, I needed to use sx on my computer, to send using the xmodem protocol as well.

The procedure to send a file through the serial using xmodem is to:

run picocom passing --send-cmd 'sx'.
in the serial (the Kindle's shell in my case), type rx <filename> where filename is the name that will be given to the received file in the destination filesystem.
press Ctrl+a followed by Ctrl+s. This will instruct picocom to send a file, which will use sx for the transfer since we passed that as the --send-cmd when running picocom.
in the prompt that appears, write the name of the file that will be transferred from the host machine, relative to the directory where picocom was run.
wait for the transfer to complete (may take a while).

Now, there's an important detail about transferring files like this that took me a while to find out, and may only be applicable to my specific setup, I'm not sure, but it is that the transfer will only work if the Kindle is charging. For some reason, it seems that when the Kindle isn't charging, the TX/RX signals aren't stable enough to the point that the checksums sent in-between the xmodem transfer aren't valid and the transfer fails. Using the shell without power connected works well enough though.

But with the charger connected and following the above steps, I was always able to transfer the files successfully.

It does take a while to transfer this way though. The transfer speed is about 10 KB/s, so the jailbreak file (~160 KB) took 15 seconds to transfer, while larger files like the KOReader package (~37 MB), which I mention below, took about one hour. Through USB the former would have been instantaneous, while the latter might have taken a couple seconds. That made me realize how far we've come in transfer speeds, so it has its charm 😝.

With this setup I was able to transfer the file and follow the remaining of the guide to successfully jailbreak my Kindle! I also installed KUAL and MRInstaller as directed in the guides so that packages are easy to install and can be accessed through a GUI in the Kindle.

Making it mine

Now that I had a jailbroken Kindle it was time to finally make it mine. The need for the .mobi format is annoying, I'd prefer to use a more standard format like epub. Fortunately that's possible on a jailbroken Kindle by using the KOReader application.

KOReader

KOReader is an e-book reader application that can be run on jailbroken Kindles (as well as other devices) and supports multiple e-book formats like epub and pdf, which is exactly what I wanted.

It's honestly great software: It is very actively maintained and improved, and has all the features I was used to in the default Kindle reader like bookmarking, dictionary search, text search and navigation through the book chapters, plus a lot more and configurations inside its various menus.

My favorite features, which are improvements over the default Kindle reader, are the progress bar at the bottom of the screen showing where you are through the book, and the fact that when the device is locked, the cover of the book is shown in the screen! 😃 This made me super happy to see. No more ads or random pictures from Amazon, just the cover of the book I'm reading. Actually, I originally installed the ScreenSavers Hack to be able to set a custom screensaver right after I jailbreaked the Kindle, but after I noticed that KOReader had this feature built-in I removed that package.

My only issue with KOReader is that sometimes it gets really laggy, so I need to restart it (which takes just a couple seconds) and it gets back to normal speed. But I should probably update it since they release a new update every month and it's already been four months since I installed it, so the issue may already have been fixed. (Yes, everything described in this post happened around April)

UART header

In order to make the Kindle surveillance-proof I had to keep the WiFi always off (also to keep the jailbreak working). And since transferring through the USB doesn't work anymore, the UART would need to be my only interface with the Kindle. This means that I need to be able to transfer books to the Kindle through the UART with relative ease. I don't consider having to open it relative ease, so I needed to make a more definitive setup.

The idea was basically to make the wires connected to the PCB pads for the UART better fixed, open a hole in the Kindle's plastic cover on the side and expose some pins there so I could easily connect a UART cable to the Kindle without having to open it.

One issue I already mentioned is that my UART cable's operating voltage is different from the Kindle's, which is why I made the converter circuit on the breadboard. I initially thought of embedding the converter circuit inside the Kindle, but realized I didn't have parts small enough for it to fit there. So I ended up just accepting that I'd need to use a converter circuit externally.

The opening in the case to expose the serial pins was done by my father! 😃

The soldering of the wire in the pads on the PCB was a bit tricky due to the very small size of the pads. With a little help from tape to fix the wire while I soldered and by choosing the right wire (those used inside Ethernet cables, they're the right amount of flexible) I managed to do it.

I was still worried about accidentally hitting the wire and disconnecting it, or even worse, irreversibly damaging the pad, so I ended up taping the wires every so often to the board, like checkpoints, and it worked wonderfully. I'll certainly keep using this technique in the future.

I left some extra length in the wire just in case it is needed in the future.

Getting the wire well connected to the headers was tricky as well. I twisted each wire around each pin, but not very tightly so the wire didn't break (this happened a few times), and added some solder. After that I added electrical tape around each pin so the wires around each pin didn't touch each other. Finally I added some more tape around all of the pins to isolate them from the gray body of the Kindle, since that's a conductive material.

After getting the pins reasonably centered in the slot, I added some durepoxi (a solid epoxy common in Brazil) and molded it all around the pins, above and below, to get them fixed to the Kindle's body.

After letting it dry overnight, the surgery was done:

Oh, and all of this took me two tries. The first time I got to the end but some pins were shorted through the gray body of the Kindle. Thankfully the epoxy wasn't completely rigid so I could remove it with some perseverance and try again.

Result

So now I have a Kindle with the UART pins exposed, this is what it looks like:

Now if I want to read a new book, I connect the charger and the converter board to it (I'm still using a breadboard for the converter circuit because I've been too lazy to make a proper board, but it wouldn't be too hard and would be a lot more compact):

I then follow the xmodem procedure I mentioned above to send the book file through UART and wait a bit for the transfer to complete...

[root@kindle documents]# rx sherlock_holmes.epub
C
*** file: ext/books/sherlock_holmes.epub
$ sx ext/books/sherlock_holmes.epub
Sending ext/books/sherlock_holmes.epub, 16078 blocks: Give your local XMODEM receive command now.
Bytes Sent:2058112   BPS:10396                           

Transfer complete

*** exit status: 0 ***
[root@kindle documents]#

This book took about 3 minutes to transfer. Usual epub book sizes are between 100 KB to 10 MB, so transferring a new book takes from 10 seconds to 20 minutes depending on the book. It's significantly more time than it would've been through USB (a couple seconds) but since I take between one and two months to finish each book it's not a problem.

When the transfer is done, all that is left to do is unhook the cables and enjoy the reading! 🙂

(I use that blue cover over the Kindle to protect it and the pins are short enough that they fit under it as well)

Note: That book is on public domain and can be downloaded for free on Project Gutenberg.

Conclusion

It took a bit more doing than I originally expected, but after all this I'm happy that I finally really own my Kindle and can have a better reading experience on it 🙂.

Blog customizations

2021-07-24T00:00:00-03:00

I've been using pelican as my static blog generator since I started this blog about one year ago. It only took a bit of configuration to get the blog up and running, and a bit of searching through pelicanthemes to find nikhil-theme which is the theme I'm still using.

While the base blog setup was easily done, it wasn't perfectly suited for my needs. This is when I started customizing a few things to make it fit my use. Thankfully pelican is really customizable, even offering a plugin interface in addition to the normal configurations. It is also written in python, so if all fails, I can always hack a patch together to get the job done 😝.

So as promised in the previous post, in this one I will talk about the main customizations I did to my blog so far. Throughout the text I'll link to the commits where I did the changes mentioned here.

Customizations

Pelican plugins

The most impactful customizations I did were probably the pelican plugins I added. These aren't my own modifications, but given their importance I want to describe them.

The two plugins I'm currently using are i18n_subsites and series, and they were really easy to find from the plugin repository.

i18n_subsites is great for my dual-language blog. By default, pelican supports only translating blog posts as standalone texts so that each translated post has links the translations. But what I really wanted was to have two versions of my blog, one in english and another in portuguese. Each with all the interface in its own language as well as only showing the posts of that language. This plugin enables exactly that.

It is a bit confusing to understand how to correctly configure it at first, mainly the template localization part, but hey, I learned about Jinja2 and gettext! Relevant commits: 1 (it was part of the initial commit, sorry), 2, 3, 4 and 5.

series I added a bit later. It's very simple but useful. In fact you can see it in action in this same post at the top. It's what enables me to make this post part of a series of posts and link to the other ones. Relevant commits: 1.

Separate RSS feed for each i18n sub-site

Also on the subject of translation, since I wanted separate sub-sites for each language, it also made sense to have a separate RSS feed for each of them. In the end this turned out to be a single line change, even though it took me a while to figure out how to do it. Relevant commits: 1.

Automatic language detection from filename

Another customization related to translations (it was a big thing for my blog! 😝) was setting the language of the post automatically from the file name. Since I already want the file for each translation of a post have the language in its name, this saves me from needing to input the language header as well.

For example, the files for this post are called 07-blog-custom-en.rst, for the english version, and 07-blog-custom-br.rst for the portuguese version. Normally I'd also need to set a lang metadata header for each one with the corresponding language, but with this change it is set automatically 🙂. Relevant commits: 1 (again, part of the initial commit 😅) and 2.

Icon setting for pages

When I was setting up the navbar on the top, I wanted to add an icon to each entry, to make it very clear what each one does. But the About page, unlike the others, is a page generated from rst and automatically added to the bar, so I couldn't set its icon in the HTML directly.

To solve this I created a custom icon metadata header that holds the FontAwesome icon name for the icon that will be displayed for the page on the bar. Then I just added the name of that "info circle" icon for the About page and now it looks much better 🙂. Relevant commits: 1.

Fix for literal rendering in rst

Let's wrap the post with the biggest change 😝.

First of all, some context: When I started the blog, I wrote the posts in Markdown. But a few months later I learned about reStructuredText (from here on, referred to as rst) because it was used in the Linux Kernel documentation, and started using it even more when I discovered rst2pdf, which I won't go into detail since I still plan on writing about it eventually. The point is that I started writing my blog posts in rst instead since it was supported by pelican.

But one really annoying issue I faced is that the default HTML generator for rst translates literal blocks into <tt> tags. This tag is deprecated in HTML5, and the whole world uses <code> for inline code, including my theme, which meant literals didn't render right. (There are reasons rst doesn't use <code> for literals, but for my blog they aren't relevant)

I could have just adapted my theme to workaround it, but that would be hiding the problem: The deprecated <tt> tag would still be there for the world to see.

At first, I worked around this issue by using the code role explicitly:

The following is a rst literal, so it will be inside a <tt> tag: ``blah``
The following uses the code role, so it will be inside a <code> tag: :code:`blah`

That is obviously a too cumbersome syntax for a blog that frequently uses inline code, so I got tired of it quickly. I then started overriding the default role to be code, and using single ticks instead of the double ticks syntax for literals:

.. default-role:: code

Now the following will use the code role: `blah`

Since the text is inside single ticks but doesn't have a role specified, it uses the default one. Way better, but I still needed to write that default-role definition at the beginning of each document...

Finally, I thought of an even better solution: Create an rst reader plugin for pelican which overrides the default reader only when parsing literals, in order to output <code> instead of <tt>.

This honestly was easier than I expected. It just took a bit of copying from pelican's and Sphinx's readers and worked perfectly! Now I could finally write plain rst literals and have them be converted to <code> tags without any additional work 🙂. Relevant commits: 1 and 2 (this is just me updating all blog texts to use the new syntax).

Conclusion

With these customizations, I have a blog I'm very comfortable with, both when writing and reading it. These changes I showed were the ones I thought were the most interesting, but naturally the whole code is open, so feel free to check the rest of it.

One year of blog

2021-06-22T00:00:00-03:00

It's been a full year since I started this blog! So I thought I'd take this chance to talk a bit about the blog itself: How it started and my thoughts on it.

Origins

I had already thought a bit about having my own blog. Having a little corner of the internet where I'd be free to talk about stuff I'm interested in sounded really cool. And maybe other people would even find it interesting as well!

But I'm not interested in web development at all, so I just wanted to write my posts without needing to deal with that. That seems to point to ready-to-use site generators like WordPress, but I didn't like the sound of that as well.

Luckily, there was already a trend of using static site generators. They were fast, simple to use and you could version the source in git and host the site for free using Gitlab/Github! As soon as I learned about them I knew that was what I wanted.

At the time two of my friends had their own blogs already (https://rafaelg.net.br and https://andrealmeid.com) using static site generators and they encouraged me to create one as well. One of them introduced me to pelican, a static blog generator in python and I decided to give it a try.

After choosing a theme and a bit of customization in pelican (which I'll cover in the next post), I was ready to start writing. And that's when I began the blog with the "Creating movie and game lists using Taskwarrior" post, which is funnily personal, while still technical. I like this style 🙂.

So that's how the blog started. Now I want to go a bit further on the principles that guide the blog.

Principles

There are two things I think are very important about the way my blog works: the dual-language and the open diary format.

Dual-language

You might have noticed that all my blog posts have a "translations" field listing one other language. And there's also a button to change languages on the navigation bar. That's because my blog is available in both English and Brazilian Portuguese. To make this possible I translated the interface to Portuguese and, more importantly, write each and every post in both English and Portuguese.

The interface was really easy to translate, but needing to write each post effectively twice is a lot of extra work. So why do I do it?

When I was thinking about creating the blog, I faced a dilemma: in which language should I write in?

On one hand, writing in English would make my content widely available, since it is the de facto language of the internet. It would make it possible for example to share a post with a community I contributed to.

On the other hand, I like my mother language. And I'd want to be able to share my posts with people I know personally without an artificial barrier: when both me and the person are more familiar with Portuguese, it seems nonsensical to transmit the information in English. I also didn't want to alienate people, from my own country, which don't speak English (even though that is becoming ever rarer).

So I decided to go with both 🙂. Double the work, but none of the downsides and double the upsides. Actually it's not double the work, since the second post is mostly a translation of the other one rather than a completely new post. The fact that I don't post that often (once per month) also helps.

Open diary format

Another thing is the way I approach my posts, and my motivation. Or the answer to the question: There are a gazillion blogs out there already, why would anyone read mine? Is it really worth it to have a blog?

And my answer to that is: Yes it is, because I'm not writing for others.

The main reason I write on my blog is because I want to record the projects I did and stuff I found interesting, so that in the future I can look back to those. So it's basically a diary (although monthly).

But of course if anyone is interested in any subject they're free to check the posts, which makes this more of an open diary. That's very cool if it happens, but I don't count on it. I write for myself, but if others like it that's a very nice bonus 🙂.

This is why I don't tend to do tutorials, and instead document my experience through learning or building something. It has a much more personal take, even though it isn't as easily approachable by others.

(After writing down these two principles I do realize that they're a bit contradictory, but what can I say, it's how I feel. I am human after all 😛.)

Since I was treating my blog as a diary it made sense to me to try and write with a fixed frequency. I ended up deciding on once a month since that was enough to keep a certain momentum while not as much as to make me stressed over it.

I also ended up making the 20th of the month be my target date. This is only a guideline. I don't force myself to do any of this. If I don't feel like posting this month or can't make it to the 20th, that's okay. But by creating this optional deadline there's a healthy pressure to make me win over any laziness and think if there was anything interesting recently that I want to share (with my future self and with others).

This has worked really well so far. I feel like I've always had something interesting to share, and I did post every single month so far, missing just a couple times the 20th (this one included 😝). By limiting myself to one post a month I also manage to create a backlog of potential posts, even though there's usually something left to do on each of them before they can be posted (I have at least three like this at the moment).

Conclusion

So, in summary, it's been a really nice journey so far. I'm happy to have already accumulated 13 posts (14 with this one) I can look back at and be nostalgic about stuff I built or learnt about, and maybe even get ideas for future projects!

I'm not sure for how long this will keep going, but so far I don't see the end 🙂.

This was it about the "human" side of the blog. In the next post I'll talk about the technical side to it: the customizations I did in the blog over this past year to suit my needs.

Keeping track of my packages

2021-05-20T00:00:00-03:00

As I previously mentioned in the "Moving to Wayland" post, I recently moved to a new computer. Moving can be very annoying if you use a heavily configured system and don't have all the configurations easily available to just move over. Since I do regular backups of my files, which include a dotfiles folder with all (or almost all) configuration files for the programs I use, I thought it would be straightforward.

While moving though, I noticed I didn't have an easy way to see and install all the programs I use. At that point I started to wonder about using a distro like NixOS that uses a declarative package manager at its heart precisely to make it easy to reproduce a system. But I like Arch Linux a lot, and there honestly isn't a reason I can't have that feature with it.

Declarative package management

So the aim was to have declarative package management to make future moves easier. Also, since then the packages are listed in a file, by versioning it in my backups I also gain the ability to rewind to an old version for free (if I want to revert some mistake in the package list, for example).

But having a simple list of the installed packages isn't the greatest thing about declarative package management. The greatest benefit is that you can organize that list in a way that makes sense to you, by reordering and grouping the packages. And you can leave comments all over the file, which is particularly useful to register the reason each package is installed, so that in the future you can use that information to remove a package when no longer needed.

A quick search on the web for declarative package managers for Arch Linux revealed aconfmgr and decpac. The former is more complete, featuring built-in configuration management as well, but that's overkill for me and it's in bash, so it's a no-go. The latter is very simple, you basically just have a file with the commands to install the packages and the packages themselves in a syntax similar to JSON. And it's written in python.

decpac also allows you to install from the AUR by relying on the yay tool. The AUR packages are identified by a simple markup before the package name.

decpac is almost perfect, but I didn't like that it did the command configuration and package listing in the same file, which made it need a more complex syntax, that is similar to JSON but not quite.

That's why I decided to create my own, which I called, very creatively, pcmn.

pcmn

pcmn is really simple, with less than 200 lines of python code. It is heavily inspired by decpac with the main difference being the simpler syntax. And it is very simple indeed:

one package per line (but lines with no packages are allowed as well of course)
everything after a # is a comment
optionally the package's group can be written inside brackets before the package name. When no group is specified the default is assumed, which installs from the official repository. The only other group is aur, which informs the package should be installed from the AUR using yay.

There are only two commands:

generate creates a new package list from the currently explicitly installed packages (that is, dependencies won't appear in the list, which is fine since pacman will resolve them).
apply applies the package list, that is, packages not listed will be uninstalled and packages in the list that aren't present will be installed.

A separate config file in JSON can be used to change the commands that are used for both the default and the aur group, to list, install and remove the packages.

I've been using it for a few months now and I'm really happy with it. As I mentioned, the main difference from decpac is that I split the configuration and the package list and I think it makes a world of difference. The package list is really clean and easy to add to. Here's a sample of mine:

#
# Utilities
#

# Backups
borg
borgmatic
python-llfuse # Needed to mount an archive

# Notifications
libnotify
mako

# Keyboard (Custom)
interception-caps2esc # Switch Caps with ESC
[aur] interception-space2meta # Use Space as Meta

Every time I notice I need a new package for something I add it to the list, together with a comment with the reason why, and run pcmn apply so that it gets installed. When I need to install a package temporarily just do something quick or some test, I install it manually using pacman as usual, so that the next time I run an apply it automatically gets uninstalled.

If you found pcmn interesting, check out its repository for some more information. It's also easily installable from the AUR, which means that after you generate the package list, [aur] pcmn-git will appear in its own package list. I find that funny for some reason 😝.

And that's why I like python so much. With less than 200 lines of code I wrote a program that solves a real problem I was having. And it was fun to write too!

Learning music theory by writing melodies

2021-04-20T00:00:00-03:00

Hey, first non-technical blog post 🙂.

Anyway, I haven't talked about this before here, but I'm really interested in playing the piano. When I was younger I took some guitar classes, but I always wanted to play the piano, and a couple years ago I finally got one! I've been playing it since, although never really took classes and also didn't practice very hard, but I do have a lot of fun with it.

What I like the most is improvising. I find it very relaxing, and sometimes interesting things come up that I want to develop further into a music.

Another thing I'm really interested in is music theory. I find it fascinating how we can dissect music to get to their building blocks and find out what makes them sound good. Also, I think if I learn more music theory I can have even more fun improvising and composing on the piano 😀.

Two great resources I'm using right now to learn music theory are:

Dave Conservatoire is like the Khan Academy for Music Theory and it's really good! I'm brushing up on the basic concepts as well as learning new ones there.

The Coursera course is what I'm most excited about, because it teaches the building blocks of music in a way that is fun, and by analyzing existing pieces of music. And even more, there are assignments in which you need to compose your own short melody in different styles. And this is what I want to talk about in this post!

So far through the course I had to make melodies in three different styles: Gregorian chant, jazz and folk. I tried to compose only by writing on the paper, but I honestly relied a lot on playing them on the piano to hear how each note sounded.

After I finished each melody, I transcribed them to a more professional music sheet using Musescore, which is an awesome open-source software for writing music sheet. What is even more awesome is that it can play your music sheet using MIDI! So for each melody I exported both the music sheet and the audio playback so I can show them both here.

My melodies

Gregorian chant

The idea for the Gregorian chant was to do something using a very limited set of notes, and few leaps, but as you can see I broke this second rule, although I think it sounds pretty good.

Being a Gregorian chant, the sheet should only have 4 lines and not have measures at all, it should have a natural flow. But Musescore isn't really meant to do that, so that's why both the audio and the sheet are more "robotic" than I'd like.

Listen here

Slow Jazz

Next, in the slow Jazz, there's should be a more strict sense of time, but have the pitches be more free, so accidentals are even encouraged.

For the accidentals I used both A and D natural, but as you can see, there aren't any sharp A's to make the natural ones feel off the key signature, as I later learned. That's definitely something I'd want to improve next time.

I also think I accidentally put too much of the aesthetic of Genshin Impact's soundtrack, which although is pretty, isn't Jazz 😛.

Listen here

Folk

Finally, the Folk melody should be something more repetitive and easy to remember and sing along. I did it in pentatonic since I learned that worked well, and I also think having an interesting rhythm is important, so that's what I did.

Listen here

These are all for now, but I hope to share more in the near future 🙂.

Moving to Wayland

2021-03-21T00:00:00-03:00

In the middle of January, my computer decided to surprise me, and not in a good way. Differently from all the quirks I've come to expect from it after all these 6 years of use — faulty keyboard, flashing screen, bad audio jack — this time it was worse, and it wasn't even a hardware issue.

At least once a day, my computer would completely freeze. The only action it responded to was, yes, holding down the power button to force it to shutdown. Not great.

I had the idea to try to SSH in using my phone (thank god for Termux) and that worked. I listed the processes and saw xorg-server using 100% of CPU. I killed it, and the computer magically came back to life. But obviously, having just killed the graphical interface, all my open applications were gone with it, and I was basically as good as getting back from a reboot anyway.

After about three days enduring this, trying to check logs to debug it, I was done with X.Org.

I'm sure most of you know about it, and I'm certainly not well qualified to talk about this, but X.Org, the application that has long powered the Linux graphical interfaces is a giant mess. Its code is very hard to maintain, and its architecture presents security concerns, to the point that a new alternative was born quite a few years ago called Wayland.

The thing is, a lot of applications today still depend on X, which also means moving to Wayland can be a breaking change. So although the move to Wayland for the ecosystem as a whole seems inevitable, people are still confortable enough with X.Org for the time being.

That was me as well, until X itself gave me a reason to quit. That issue could probably be investigated and fixed, by why waste time on something that will soon enter maintenance mode? It was finally time I gave Wayland a shot.

Making the switch

From my point of view, switching to Wayland means essentially installing it and changing all the applications I used on X, to equivalent ones that work on Wayland. The main one being the graphical interface itself, which in my case was the i3 window manager.

Thankfully, there's an already very well established i3-compatible window manager for Wayland called sway, so that change was pretty straight-forward. It was just a matter of installing sway and changing the config a bit.

Something that helped things a lot was that the sway wiki has a i3 migration guide, which not only shows the main changes needed in the i3 configuration for it to work on sway, but also a list of the Wayland programs equivalent to the ones normally used on X.Org.

For instance, the keyboard layout setup and toggling, which is normally handled by setxkbmap in X, is integrated in sway so I just used input commands in sway's config to configure the possible layouts and to have a binding for toggling it.

Still on the keyboard topic, I have grown really accustomed to making my Caps Lock work double duty: it acts as Esc when just pressed independently, and as Control when held while pressing another key. I used setxkbmap together with xcape to achieve this, but those obviously aren't an option on Wayland.

To my rescue came Interception Tools, which allows the same to be done regardless of being on X or Wayland. In itself it just provides the framework, but the function of mapping both Esc and Control to Caps Lock I mentioned can be easily configured by using the official plugin caps2esc.

Oh, and it can also be combined with another plugin, space2meta, to use the Spacebar key as space when just pressed, but as the Meta key (the modifier on i3/sway) when held. I had previously experimented with this in X using xcape but I kept accidentally triggering bindings, so reverted back. But since space2meta is smarter and only sends it on the key release event, it's really usable and worth it for the increased comfort.

A bit of a side track: If caps2esc and space2meta sound like crazy talk to you, then I do urge you to try them out. Mainly caps2esc, as that one doesn't have any drawbacks, honestly. Like, how frequently do you use Caps Lock anyway? There's no reason to keep it there in the home row. Esc and Control on the other hand are really useful, even more if you use Vim. Now back on track...

What about rofi, a menu launcher that I use for all sorts of stuff? It's also X dependent. There's wofi as an alternative for Wayland, but I didn't really like it. Felt a lot slower and I'm not a fan of it being GTK (the magnifying glass icon beside the input field really bothers me). I just want a single color rectangle with a prompt and options highlighted below, and rofi does that much better. Luckily some brave soul created a rofi fork for Wayland and it works like a charm. Mostly...

Unfortunately, the window switching functionality doesn't work. And I find it quite useful to go directly to a specific open application, otherwise I need to cycle through all workspaces searching for it. But it was very easy to find a window switcher script on the web, although for wofi, so I simply changed it a bit to work with rofi and got myself a window switcher script. And that's a perfectly fine rofi in Wayland.

Taking screenshots with maim was also no longer possible. The alternative is grim, and a separate program, slurp, to be able to select a region or a window for the screenshot. As a nice bonus, as grim's README shows, it's very simple to make a color picker using it together with slurp and imagemagick, so I set that up as well. My previous attempt at doing this with maim failed, so this was a nice bonus of moving to Wayland.

xclip obviously also doesn't work on Wayland. So to keep my clipboard working, I needed to install wl-clipboard. This wasn't enough for Vim though. In Vim, operating on the system clipboard is done through the + register, like "+yy to copy the current line to the clipboard, but that doesn't work with Wayland's clipboard. I got really close to installing Neovim since it seems to work there, but I found the vim-wayland-clipboard plugin for vim which works flawlessly.

One thing I didn't expect to not work out of the box was screen sharing. Thankfully Pipewire's page on ArchWiki gave instructions on how to get it working, which was pretty simple too: just set an environment variable and install xdg-desktop-portal-wlr and pipewire-media-session. That said, I did have an issue after a pipeWire update, but it seems to have been solved since.

I also changed my notification daemon from dunst to mako. That said, as of version 1.6.0, dunst supports Wayland as well although it still isn't packaged on Arch Linux. But I'm fine with mako too to be honest, so will probably stay with it.

Some other changes that I don't think deserve any special comment:

light instead of xbacklight to control the screen brightness
gammastep instead of redshift to make the screen temperature more comfortable at night
imv instead of sxiv as the image viewer

And finally, it's important to mention Xwayland. It's the application that allows you to run X applications on Wayland. It doesn't work for everything (otherwise I wouldn't have changed all those previous applications), but for example for Electron applications and for the Steam and games it has worked flawlessly so far to that point that it's hard to know that they aren't native on Wayland, which I think is awesome. If Wayland is to be the future, it has to allow current X applications to continue to run on it until they are properly ported over to Wayland.

Plot twist

After all that change I was all set up on Wayland, but there were actually some really annoying graphical glitches. From time to time, applications would start flashing and glitching and there wasn't anything that made it stop, except, of course, quitting Wayland. So I was essentially where I started, although a bit better since in this case the whole computer didn't freeze, so I could at least save everything before rebooting.

In the meantime though, I got a new computer from my employer, which was much appreciated given all the quirks of the old one. Since I had already gone through all the trouble of the Wayland transition, I installed sway in this new computer as well. And everything worked perfectly. No issue whatsoever.

So, what is the conclusion of this story? The crashing/glitching issue was something related to my old computer's hardware rather than X itself, but at least that gave me the push I needed to migrate to Wayland. I could have probably stayed on X on the new computer, yes, but that would only be delaying the inevitable. The important thing is that now everything works, and I feel much more comfortable debugging and reporting issues with the graphical interface, given that it's actively maintained, if that ever happens in the future.

Ledsticker: My first holistic SW + HW project

2021-02-20T00:00:00-03:00

It was in the middle of 2019. I was taking classes in Embedded Systems Laboratory and had to make the final project of my choice. I had the idea to do some sort of a Guitar Hero using a keyboard for the input, an 8x8 LED matrix to display the notes and speakers to play the note sounds. The problem was that the university didn't offer the LED matrix or the speakers, so I decided to buy them, with the excuse that I would want to use them later for my personal projects as well.

That project went well, and the end result can be seen on YouTube. Having completed it, I started thinking about what to do with the LED matrix. Eventually, I had an idea I thought was so cool that it just had to be done.

I enjoyed adding cool stickers to my notebook's lid, and had a lot of them, but when you get down to it, stickers aren't that interesting. So what if I used the LED matrix as a sticker? I could glue it to the notebook lid and connect it to the USB port, so it could get power as well as be controlled. I would be able to program anything I wanted to be shown on the matrix.

Prototyping

The first step in turning this into reality was to build a prototype. I already had the LED matrix module, but it received data from an SPI bus, rather than the USB that I wanted to use. I started searching the internet for a circuit that converted USB to SPI and found the MCP2210 IC. I bought some those, but also bought its development board version, with the IC on a board ready to use, to make the initial prototyping easier.

I then searched for software that implemented the interface with MCP2210 and found MCP2210-Library. I started writing my own software to initialize the IC responsible for controlling the LED matrix, MAX7219, (present in the LED matrix module) by calling the library functions to send the SPI data through USB (which MCP2210 then forwarded through the SPI bus to MAX7219).

At this point I had the minimum necessary to verify whether the project was viable: the necessary hardware components in their own separate ready-to-use modules, and a minimum software capable of interacting with the ICs and drawing a fixed pattern on the LED matrix. I connected the USB-to-SPI module to the computer and to the LED matrix module and ran my program:

Having confirmed that the project was indeed viable, I started advancing on the necessary steps to make it usable for the end user. The first thing was to make it easy to control the LED matrix for the intended purpose. This is when I decided on the sticker concept, which is central to the project.

Stickers

I wanted it to be possible to show three different things on the LED matrix:

A static "image", that is, turn a static pattern of LEDs on.
An animated "image", that is, show a sequence of patterns, each one after some delay.
An animated "image" but that was dynamically generated from the output of a program, so I could show a simulation of the Game of Life, for example.

To enable the two first use cases, I defined the concept of a static sticker. It is simply a text file containing a sequence of commands that determine what is shown on the matrix. There are commands for turning individual pixels, whole columns or rows, or the whole screen on or off, for updating the screen, etc.

To enable the third use case, I defined the dynamic sticker, which is any program, programmed in any language, that outputs those sticker commands. In this way, a programming language can be used to generate a more complex and dynamic sticker.

So the idea is that the user would execute ledsticker and pass either a static or dynamic sticker as a parameter. If it was static, the commands would be read and the LED matrix updated accordingly. Otherwise if it was dynamic, it would instead be executed and its output used as a static sticker in the same way.

For all of this to work, I had to both define all sticker commands, as well as implement a simple parsing for them in my program. As an example of the commands, the simplest one is on R C, which turns the LED at row R and column C on.

Finally, to make sure those commands were enough, and also for fun, I implemented some stickers: a single frame with the Creeper face, an animated Tetris, and a dynamically generated simulation of the Game of Life.

Hardware

At this point I had the software fully operational, so I wanted to start tackling the hardware side. I was using separate modules for the LED matrix and the USB-to-SPI, but the objective really was to have it all in a single small PCB, in order to be convenient to attach it to the notebook lid.

But before making the PCB, I had to be sure about the circuit. I had the two separate modules which were known to work, so I used them as a reference and read the datasheets for MAX7219 and MCP2210 in order to create the circuit schematic in KiCad.

I ended with a schematic containing the two main ICs, MAX7219 and MCP2210, as well as all the other passive components needed to make them work (resistors, capacitors and a crystal), together with the LED matrix and a USB mini connector.

The schematic symbol (and footprint) for the LED matrix (model 1088AS) in particular I had to create myself, since it didn't exist. I followed DigiKey's Intro to KiCad series for that.

With the circuit decided, I soldered all the components and connections in an universal board (would've been easier on a breadboard, but I don't have a good one) to test. The most difficult part of this was that the MCP2210 IC is a Surface Mount Device (SMD), and SMDs aren't meant to be soldered on universal boards. What I did was to order a SMD adapter PCB with the correct pin sizes so I could solder the IC on it and solder it on my board. This is what it all looked like:

After testing this board with some stickers and concluding that it was working fine, the only final thing to do was the PCB.

PCB

I had never made a PCB before, and the idea of finally learning it was one of the things that motivated me to make this project in the first place.

I learned almost everything from following the DigiKey's Intro to KiCad series I already mentioned. There are a lot of different steps in making a PCB, but they aren't really difficult, just laborious sometimes. Positioning the components in a compact way while also minding the traces that had to be made was very tricky, but it was a fun puzzle.

After finishing the project in KiCad, I ordered the PCB from OSHPark. After 3 long months waiting for it (due to overseas shipping delay due to the pandemic), the board finally arrived.

After waiting so long for the board I didn't waste a second to solder. I got all the components and soldered everything on the same day. Here are all the components next to the PCB:

Halfway there (or so I thought):

Soldering the SMD components (MCP2210 and USB mini connector) was a lot harder than the rest. The USB mini in particular was nearly impossible. The pads for its 5 pins were almost the same size of the leads themselves, and they weren't very easy to reach with the soldering iron. After multiple tries, I finally got it though. Here is the final board:

Needless to say that it was incredible to see my very own PCB. There's something magical about holding something you designed yourself, that writing software could never provide. Seeing the whole circuit tidily organized in this PCB also felt so much more robust and professional (look at the prototype board again for comparison...).

Results

With everything finally done, the board, and the program, it was time to play. The following gif shows both the command line used to load the sticker and the ledsticker board being updated. Three stickers are loaded one after the other. First a static Creeper face, then a Tetris animation, and finally a Game of Life simulation generated from a python program "in real-time" [1]:

And this marks the project as complete! Wait, but what about gluing the board to the notebook lid? Well, when I started this project, I intended to do that due to a social reason: as I walked with my notebook around the university, people would ask me about it, so I would have an excuse to geek out about it and it would be an interesting conversation starter. With the pandemic, I'm always at home, so doing it would not only be pointless, but then even I wouldn't see it. That's why I'm postponing the attachment to after I'm back going to public spaces again. But just to give an idea of how it would look like, I attached it using tape just for this picture:

Conclusion

This was by far my favorite project I have ever done. It was really a holistic project, where I needed to design both the software and hardware and think about so many different aspects. On the software side I had to create commands to provide a good interface with the user as well as implement the program as a whole (except for the communication with MCP2210). On the hardware side I investigated datasheets to design the circuit, arranged it all to be compact and also created my first PCB!

If you found the project interesting, I invite you to make your own ledsticker board and create your own stickers! Everything is open. You can find the command line program as well as more detailed information on it at ledsticker. The board schematics together with the hardware description can be found at ledsticker-hw. You can also order the PCB directly from OSHPark if you want (I won't receive anything from it). If you do make the board or a sticker, let me know! 😃 (You can find my email in the About page).

[1]	It isn't really real-time. The dynamic sticker program is run as fast as it can, filling the input buffer of `ledsticker`, which is consumed frame by frame a lot slower, depending on the FPS set.

Organization beyond Taskwarrior

2021-01-20T00:00:00-03:00

In the previous article of this series, I went into all my Taskwarrior and VIT customizations, and my workflow with them, that enables me to organize my tasks and get them done. Tasks, however, aren't the whole story when getting organized.

Another crucial component of organization is having a calendar. It enables you to be aware of time-sensitive tasks and events, and also to make informed decisions when scheduling new events. Of course this is nothing new, and is even part of GTD.

Something that isn't in GTD, though, and that I missed, is something to keep track of a schedule. GTD is a great system to keep track of all moving parts of projects in your life, and to get them done, but it isn't at all concerned with reserving different portions of your day or week to do the basics (like eating), doing something regularly, or advancing in your general tasks. It just organizes what to do and in which contexts, not precisely when, which may be lacking, if one is trying to use their time well.

So, besides having VIT as my central tasks organizer, my system also needs

a decent calendar, somehow integrated with VIT to also show due dates of tasks, in addition to normal appointments
a way to set a schedule, and to be constantly reminded of it

Let's go over each one.

Calendar using calcurse

First of all, Taskwarrior actually does have a calendar, which can be shown with task calendar, but it's honestly useless. You can only see which days have tasks, but not which tasks those are.

I wanted a calendar that gave me a good monthly and weekly overview of appointments, that was lightweight (preferably on the terminal), and customizable enough to be integrated with VIT. I ended up using calcurse.

Now, the big question was: How can I have my tasks show up in calcurse? Well, since calcurse uses a text file to hold all appointments using a simple syntax, I just needed a script that reads all tasks from Taskwarrior and outputs the appointments using calcurse's syntax. And this is the script I wrote:

#!/bin/python

from tasklib import TaskWarrior
import sys

tw = TaskWarrior('/home/nfraprado/.task/data')

# Parse appointments
apts = tw.tasks.filter('(status:pending or status:waiting or status:completed)', type='cal')
for apt in apts:
    start = apt['scheduled']
    if start is None:
        sys.stderr.write(f"Apt '{apt}' has no sched date!\n")
        continue

    summary = str(apt)

    if start.hour == 0 and start.minute == 0:
        start_fmt = start.strftime("%m/%d/%Y")
        print(f"{start_fmt} [1] {summary}")
    else:
        start_fmt = start.strftime("%m/%d/%Y @ %H:%M")

        if apt['due']:
            end_fmt = apt['due'].strftime("%m/%d/%Y @ %H:%M")
        else:
            end_fmt = start_fmt

        print(f"{start_fmt} -> {end_fmt}|{summary}")

# Parse due dates for next actions and projects
tasks = tw.tasks.filter('(status:pending or status:waiting) and (type:next or '
                       'type:objective or type:standby)')
for task in tasks:
    for date_type, label in [('due', "Prazo final: "),
                             ('scheduled', "Prazo inicial: ")]:
        if not task[date_type]: # Skip tasks with no date
            continue
        start = task[date_type]

        proj = "Projeto: " if task['type'] == "objective" else ""

        summary = label + proj + str(task)

        if start.hour == 0 and start.minute == 0:
            start_fmt = start.strftime("%m/%d/%Y")
            print(f"{start_fmt} [1] {summary}")
        else:
            start_fmt = start.strftime("%m/%d/%Y @ %H:%M")
            end_fmt = start_fmt
            print(f"{start_fmt} -> {end_fmt}|{summary}")

This script is basically concerned with which tasks will be shown in the calendar, and in which format. First, there are the cal tasks, which, if you remember from last post, are my appointments, and they are the main thing that should be shown in the calendar. Each one of them is converted into an entry for calcurse, with the scheduled date being used as the start time, and the due date as the end time. The appointment text is just the plain task description.

Then, there are also the normal tasks, which I want to show not as a continued event in the calendar, but rather I want to have an entry for the start date, to show me when I can start doing the task, and another one for the due date, to show me before when it needs to be done. I also want custom labels prepended to these entries, so I can tell them apart from the appointments. So this script prepends "Prazo inicial: " to the scheduled date entry, and "Prazo final: " to the due date entry. Additionally, if the task is an objective task, "Projeto: " is also added to the description's entry, meaning the date is relevant for the whole project rather than a single next action task.

Initially, that was it. I mapped a key in VIT to run this script, and then reloaded calcurse's appointments with R. So, I needed two key presses in two separate windows to see the calendar updated.

I later discovered that calcurse also supports hooks (like Taskwarrior, as shown in the previous post), and added a pre-load hook with the following:

taskwarrior-task2cal > /home/nfraprado/.calcurse/apts

Meaning when I type R in calcurse, it automatically runs my script to export the tasks to calcurse's file, so it is now a single key in calcurse to see my calendar updated! 🙂

The following gif shows both a next and a cal task being added in Taskwarrior and automatically showing up in calcurse:

At asciinema.org

Another little thing is that I have calcurse's notification.command configured with the following:

calcurse --next | sed -n -e '2s/.*\] \(.*\)/\1/p' | xargs -I '{}' notify-send '  Upcomming appointment' '{}'

This makes so that it shows a notification in my system some time (configurable, I use 10 minutes) before every appointment, with its description.

Schedule using python

The first step in maintaining a schedule is, of course, to create it.

I wanted a simple and easy way to define and later edit my schedule, so I implemented it using dictionaries in python, with the schedule of each day of the week being given by a separate dictionary.

The idea is that the key gives the start time of an action, and the corresponding value is the action itself. The action is considered the current one from that time until the time of the next action. For example, I have the following base schedule dictionary:

base = {
    '08': "Banho+café",
    '12': "Almoçar",
    '15': "Piano",
    '19': "Jantar",
    '24': "Dormir",
}

If this were used as a schedule, it would mean that the schedule starts with "Banho+café" from 8 in the morning until 12 PM, when it turns into "Almoçar", and so on.

Like any python dictionary, I can then extend it to implement the schedule of a day, like

segunda = dict(base)
segunda.update({
    '09': "Tarefas",
})

Now, considering segunda as the schedule, "Banho+café" only goes until 9 AM, when it turns into "Tarefas", which in turn goes until 12 PM, when "Almoçar" starts, like before, and so on.

A value can also be deleted, like always, using del segunda['09'], for example.

To define my weekly schedule I just need to create one dictionary for each day of the week using specific variable names (segunda, terça, quarta, quinta, sexta, sábado and domingo).

I like this system because I can add actions simply adding its name and start time, and also because I can add common actions to a base dictionary that is extended by each day's dictionary, reducing redundancy.

Next, I have a python module that knows how to parse each dictionary to return the information I'm interested in:

import datetime

import cur_sched


scheds = [cur_sched.segunda, cur_sched.terça, cur_sched.quarta,
          cur_sched.quinta, cur_sched.sexta, cur_sched.sábado,
          cur_sched.domingo]

gran = 30
max_minute = 60 - gran


def get_cur_wday_time():
    weekday = datetime.datetime.today().weekday()
    hour = datetime.datetime.today().hour
    minute = (datetime.datetime.today().minute // gran) * gran
    if hour == 0:
        hour = 24

    return weekday, hour, minute


def format_time(hour, minute):
    if minute > 0:
        time = f"{hour:02}h{minute:02}"
    else:
        time = f"{hour:02}"

    return time


def get_current():
    weekday, hour, minute = get_cur_wday_time()
    return get_sched(weekday, hour, minute)


def get_sched(weekday, hour, minute):
    for m in range(minute, -1, -gran):
        try:
            return scheds[weekday][format_time(hour, m)]
        except:
            pass
    for h in range(hour - 1, 1, -1):
        for m in range(max_minute, -1, -gran):
            try:
                return scheds[weekday][format_time(h, m)]
            except:
                pass
    return ''


def get_new():
    weekday, hour, minute = get_cur_wday_time()
    try:
        return scheds[weekday][format_time(hour, minute)]
    except:
        return ''

get_current() returns the current schedule action based on the current time. get_new() does the same, but only if the action just started. For example, if "Piano" goes from 3 to 4 PM, and considering a granularity of 30 minutes (which I'm currently using), it will return "Piano" only between 3 and 3:30 PM.

To always be able to easily see what's the current action based on my schedule, I have an i3blocks block in my status bar specific for it:

It just calls get_current() from the previous python module.

But only knowing the current action isn't enough, I need to be notified when the current schedule changes. That's why I also have a cron job running every 30 minutes and calling get_new() to check if the scheduled action changed and if so, showing me a notification:

Finally, it's also useful to see the weekly schedule as a whole sometimes, so I have a script that prints it, using a different color for each action in the schedule, with the colors themselves being random (so they change on every new run):

#!/bin/python

import schedule
import colored

weekday_name = ["2a", "3a", "4a", "5a", "6a", "Sáb", "Dom"]
color = True

print("      ", end='')
for weekday in range(0, 7):
    print(f"{weekday_name[weekday]:15}", end='')
print()


def get_color(text):
    hex_num = hex(hash(text) % (16 ** 6))
    hex_num6 = hex_num[:2] + hex_num[2:].rjust(6, '0')
    return hex_num6.replace('0x', '#')


for hour in range(8, 25):
    for minute in range(0, schedule.max_minute + 1, schedule.gran):
        if hour == 24 and minute != 0:
            break
        print(f"{schedule.format_time(hour, minute):5}" + " ", end='')
        for weekday in range(0, 7):
            text = schedule.get_sched(weekday, hour, minute)
            if color:
                print(colored.stylize(f"{text:15}", colored.fg(get_color(text))),
                      end='')
            else:
                print(f"{text:15}", end='')
        print()

Extra: tasks on the status bar

Since I already use i3blocks as my system's status bar, I also added a block with task information to help me to stay on top of my tasks and to regularly review them (and not only during the weekly review):

The string in the beginning shows by current context, in this case, sp. The three numbers following are the number of pending in tasks (in yellow), the number of "stuck" projects (in magenta) and the number of tasks due this week (in red).

Future improvements

And that's all there is to my organization system. It's basically VIT on top of Taskwarrior to organize my tasks, calcurse to show my calendar, and blocks in the status bar and notifications to help me track and to warn me about my tasks, schedule and appointments.

This system serves me well, though there are still things to improve. Mainly integration with my phone. As I previously noted, this isn't an issue right now since I'm always home, but as soon as the pandemic is over, I need a good way to have my tasks on my phone synced with my computer. I need to at least be able to easily add in tasks, and see all of my reports, optionally with some filter. I will also need a calendar with the same integration with Taskwarrior I have on my computer. Perhaps with the whole "Convergence" theme going on with Purism, I may end up buying a Librem 5 and having a similar setup on both devices 😃.

Managing my tasks using VIT

2020-12-22T00:00:00-03:00

Two years ago I decided to get more organized about my life. During that time I read the Getting Things Done book and discovered Taskwarrior, a task manager for the terminal which doesn't get in the way.

I greatly appreciated Taskwarrior's simplicity and customizability, but after some time, the need to write a command for every single action, like task list to list the tasks, gets tiresome. Even after using aliases to shorten the commands, like tl for task list.

After searching for a TUI for Taskwarrior, I found VIT. I actually only stuck with VIT on the second try, because when I first found it, it was written in Perl and wasn't that great of an interface. But with the rewrite in python by thehunmonkgroup and the release of VIT 2.0, it became the perfect interface for Taskwarrior.

Before I start explaining my setup, keep in mind that I won't go into detail of GTD itself, so if you're not familiar with it, maybe take a look at GTD in 15 minutes. Things will make more sense.

Taskwarrior configuration

To organize my tasks following the GTD method, I need to add some custom attributes (called UDAs) and reports in my Taskwarrior configuration file (.taskrc).

My UDA definitions are the following:

uda.type.label = Type
uda.type.values = in,next,objective,someday,standby,cal

uda.priority.values = H,L,
urgency.uda.priority.H.coefficient = 6.0
urgency.uda.priority.L.coefficient = -6.0

uda.difficulty.type = string
uda.difficulty.label = Difficulty
uda.difficulty.values = H,L,

This adds three different attributes. The first and most important is the type attribute. I use it to assign the task to one of the main lists defined in GTD:

in assigns the task to the "In" list, where I first collect my tasks to take them out of my head.
next puts it in the "Next actions" list, where the tasks I need to be doing next live.
objective assigns it to the "Projects" list, where each task describes the objective of one of my current projects, and guides the tasks I create on "Next actions" for each one of the projects.
someday puts the task in the "Some day/maybe" list. Tasks put there don't need to be done anytime soon, and may even be uncertain ideas that won't ever be done at all. Whenever I'm sure I don't want to do it, however, I delete the task.
standby assigns it to the "Waiting for" list. That's where the tasks that depend on the action of others sit.
cal puts the task in a "Calendar" list, so that it appears on my calendar with the right date set.

The other attribute is priority, which I use to prioritize tasks. No priority means medium priority. H means "High" and L means "Low", and they increase and decrease the task's urgency, respectively. The next report sorts tasks by urgency, so setting a high priority makes the task appear higher on the report, which makes it draw more of my attention as I skim on the report top to bottom looking for what to do next.

The last attribute is difficulty which tracks the difficulty of the task, that is, how much energy it would cost me to complete it. At the time of this writing, I seldom use it, but the idea is so I can, for example, filter the next report with only the L difficulty tasks (i.e., the easy ones) whenever I'm tired.

I also think about adding a duration UDA for tracking, also with H or L, if I expect a task to take a lot or little time, so I can filter on however much time I have available at the moment. I still haven't find the need for it though.

Then, there are the reports:

report.next.columns = id,start.age,priority,project,tags,recur,scheduled.countdown,due.relative,until.remaining,description.count,urgency
report.next.labels = ID,Active,P,Project,Tag,Recur,S,Due,Until,Description,Urg
report.next.sort = urgency-

report.all.columns = id,status.short,uuid.short,start.active,entry.age,end.age,type,depends.indicator,priority,project,tags.count,recur.indicator,wait.remaining,scheduled.remaining,due,until.remaining,description
report.all.labels = ID,St,UUID,A,Age,Done,Type,D,P,Project,Tags,R,Wait,Sch,Due,Until,Description

report.all_valid.columns = id,status.short,uuid.short,start.active,entry.age,end.age,type,depends.indicator,priority,project,tags.count,recur.indicator,wait.remaining,scheduled.remaining,due,until.remaining,description
report.all_valid.labels = ID,St,UUID,A,Age,Done,Type,D,P,Project,Tags,R,Wait,Sch,Due,Until,Description
report.all_valid.filter = (status:pending or status:waiting)

report.in.columns = id,description
report.in.description = Inbox
report.in.filter = status:pending limit:page (type:in)
report.in.labels = ID,Description

report.someday.columns = id,description.count
report.someday.description = Someday/Maybe
report.someday.filter = limit: type:someday status:pending
report.someday.labels = ID,Description

report.standby.columns = id,priority,project,due.relative,description.count,urgency
report.standby.description = WaitingFor
report.standby.labels = ID,P,Project,Due,Description,Urgency
report.standby.filter = limit: type:standby status:pending +READY
report.standby.sort = urgency-

report.objectives.columns = id,priority,project,description.count,urgency
report.objectives.description = Projects
report.objectives.labels = ID,P,Project,Description,Urgency
report.objectives.filter = limit: type:objective status:pending +UNBLOCKED
report.objectives.sort = urgency-

report.type.columns = id,description,type
report.type.description = Type
report.type.filter = status:pending limit:page
report.type.labels = ID,Description,Type

report.cal.columns = id,entry.age,recur.indicator,scheduled,due,description
report.cal.description = Calendar
report.cal.filter = type:cal status:pending limit:page
report.cal.labels = ID,Age,R,Scheduled,Due,Description
report.cal.sort = scheduled

There is a report for each of the aforementioned types so that I can see the list of tasks of each of them: in, next, objectives (with an 's'), someday, standby and cal. Additionally, there's the all report which shows all the tasks, the all_valid report which is like all but hides the completed and deleted tasks, and the type report which shows each of the tasks and its type.

VIT configuration

If configuring Taskwarrior is all about setting up the attributes and reports to enable my workflow, configuring VIT is all about setting up bindings to make the workflow as fluid as possible.

My bindings (which are set in the config.ini file inside the .vit folder) are the following:

q = {ACTION_QUIT}

a = {ACTION_NOOP}
aa = {ACTION_TASK_ADD}
ai = aatype:in<Space>
an = aatype:next<Space>
ap = aatype:objective project:
as = aatype:someday<Space>
aw = aatype:standby<Space>
ac = aatype:cal schedule:
aN = aatype:next project:{TASK_PROJECT}<Space>
aP = aaproject:{TASK_PROJECT}<Space> 

gi = :in<Enter>
gn = :next<Enter>
gp = :objectives<Enter>
gs = :someday<Enter>
gw = :standby<Enter>
gc = :cal<Enter>
gl = :list<Enter>
ga = :all_valid<Enter>
gA = :all<Enter>
gP = :all_valid project:{TASK_PROJECT}<Enter> 

M = m{TASK_DESCRIPTION}
S = :!r task modify type:someday {TASK_UUID}<Enter>
W = :!r task modify type:standby {TASK_UUID}<Enter>
P = :!r task modify {TASK_UUID} priority:
F = :!r task modify {TASK_UUID} difficulty:
Y = :!r task duplicate {TASK_UUID}<Enter>

#$ = :!r task sync<Enter>
zp = gpf{STUCK_PROJS}<Enter>
o = :! taskopen {TASK_UUID}<Enter>
C = :!r taskwarrior-update_context<Enter>

# Convenience mappings
1 = :1
2 = :2
3 = :3
4 = :4
5 = :5
6 = :6
7 = :7
8 = :8
9 = :9

- = m-
+ = m+

The main bindings are the ones starting with a or g. Those are the ones I use all day. The ones starting with a are for adding tasks, and there's one for each task type, so I can quickly add a task of any type. The ones starting with g are for going to a report, so I can also quickly jump to a specific report and see the tasks of that type.

Some of those are a bit special though. For example, aN adds a task of type next and with the project of the currently selected task. I use this when I'm reviewing my current projects on the objectives report and want to add a task for the next action of the project I'm currently selecting.

gP is another one I use a lot. It shows all tasks with the same project of the currently selected task. I use this when I'm looking at a task from a project and want to see all other tasks of that project, like the objective given by the objective task, what are the next tasks for it, if there are things waiting on other people at standby or something marked on the calendar at cal.

Then there are some utility bindings used less often. M edits the selected task description. S and W change the selected task's type to someday and standby, respectively. The former is useful for when I decide a task should be done in the future rather than now, while the latter isn't used much. P edits the task's priority, while F edits the task's difficulty. Y duplicates the selected task.

The $ that is commented out is used to sync the tasks to a central Taskwarrior server using Taskserver. It was essential to keep the tasks synchronized between my computer and my phone when going out. But since going out hasn't been a common theme recently, meaning I'm always on my computer, I disabled it for the time being.

The zp binding is one that is a bit more complicated but very handy. It goes to the objectives report, that shows my current projects, and filters so that only the projects with no next tasks are shown. It is important with GTD to always make sure that all your projects have next actions assigned to them, so that the next step in advancing them is obvious. With this binding, during my weekly review of all the tasks, I can easily see the projects with no next actions and then create one for each project using the aN already shown. If you're wondering what {STUCK_PROJS} means, don't worry, I'll explain it shortly.

o uses taskopen on the currently selected task, and this is another one I use all the time. What taskopen does is read through the task's annotations and open one of them (asking which one, if there are multiple options). If the annotation is an URL, it will be open on the web browser. If it is the string "Notes", taskopen will open the text file associated with that task (or create one if this is the first time) where you can write longer annotations. These two are the ones I know and that I use all the time. Normal text annotations are ignored by taskopen.

The C binding runs a script to update the current Taskwarrior context, but I normally don't need to run it since it runs automatically as explained in the "Automatic context detection for Taskwarrior" post.

Finally, I have some convenience mappings. Each of the digits maps to : followed by that digit, so I can jump to a task with one less keystroke. In more detail, normally to jump to task 42 I would need to type :, 4, 2 and <Enter>. With this binding I can skip the :, so I type just 4, 2 and <Enter>. Since it is very common to jump to tasks in VIT, this one less keystroke pays off. Additionally, - and + map to m- and m+, respectively, where m is the default command to modify the task, so to add a tag to a task I just press +, type the tag name and press <Enter>.

VIT variable replacements

You may have noticed some {SOMETHING} in the VIT bindings above. I just wanted to give a short explanation on those (since a full explanation should be read in VIT's documentation) and also show my custom variable replacement.

First things first, the {ACTION_QUIT} in the q binding isn't even a variable replacement, although the syntax is similar (the difference being that it is the only thing after the =). That's just one of VIT's actions that can be mapped. A variable replacement occurs in the aN binding for example:

aN = aatype:next project:{TASK_PROJECT}<Space>

Here, {TASK_PROJECT} will be substituted by the project attribute of the currently selected task. So that's why that binding does what it does. The aa in the beginning is mapped to the action to add a new task, then the type is set to next and the project to the selected task's project. All {TASK_*} are built-in VIT variable replacements, and can be used for any task attribute (including UDAs).

Now, in the case of the zp binding:

zp = gpf{STUCK_PROJS}<Enter>

{STUCK_PROJS} is a custom variable replacement that I created. It was very simple, I just followed VIT's CUSTOMIZE.md.

Inside my .vit folder I added a keybinding/keybinding.py with the following:

from task_proj_stuck import get_stuck_proj_ids

class Keybinding():
    def replacements(self):
        def _custom_match(variable):
            if variable in ['STUCK_PROJS']:
                return [variable]
        def _custom_replace(task, arg):
            if arg == 'STUCK_PROJS':
                return ' '.join(list(get_stuck_proj_ids()))
        return [
            {
                'match_callback': _custom_match,
                'replacement_callback': _custom_replace,
            },
        ]

I then have a task_proj_stuck python module with the following:

from tasklib import TaskWarrior

tw = TaskWarrior()

def get_stuck_projects():
    """ Get taskwarrior projects that don't have any next actions assigned to
    them.  Next actions here mean actions of type 'next', 'standby' or 'cal',
    either pending or waiting. """

    projects = tw.tasks.pending().filter('+READY', type='objective')

    next_tasks = tw.tasks.filter('(status:pending or status:waiting) and type:next')
    standby_tasks = tw.tasks.filter('(status:pending or status:waiting) and type:standby')
    cal_tasks = tw.tasks.filter('(status:pending or status:waiting) and type:cal')

    for project in projects:
        count_next = len(next_tasks.filter(project=project['project']))
        count_standby = len(standby_tasks.filter(project=project['project']))
        count_cal = len(cal_tasks.filter(project=project['project']))
        if count_next + count_standby + count_cal == 0:
            yield project


def get_stuck_proj_ids():
    return (str(project['id']) for project in get_stuck_projects())

So what happens is that the get_stuck_proj_ids() function returns a generator containing the id of each objective task whose project doesn't have any next tasks. The {STUCK_PROJS} custom variable replacement then just calls this function and joins the ids with space.

For example, suppose there's project clean-bedroom and project write-vit-post. Project clean-bedroom's objective task has id 42 and project write-vit-post's objective task has id 99. Both of these projects don't have any next tasks, while all other projects have. Then, by pressing zp, VIT will execute gpf42 99<Enter>, which goes to the objectives report and filters for just tasks 42 and 99, so I can focus on adding next tasks for each one of these stuck projects with aN. Pretty convenient, right?

Hooks and taskpirate

Another powerful feature of Taskwarrior that enables extensibility is the hooks API. If you use Git, you may already be familiar with this concept. It enables a custom script to be run when a certain event occurs in the program, in this case, in Taskwarrior.

Instead of just creating a Taskwarrior hook directly, I decided to use taskpirate, which makes the tasks more directly accessible in python. And as you may already know, I like python.

I currently have a single hook named pirate_add_inherit.py, inside the hooks folder, and what it does is to make certain attributes inheritable from the objectives task of a project. The code is the following:

#!/bin/python

from tasklib import TaskWarrior

def hook_inherit(task):
    if task['project'] and task['type'] != 'objective':
        tw = TaskWarrior('/home/nfraprado/.task/data')
        try:
            obj_task = tw.tasks.filter(type='objective', project=task['project'])[0]
        except:
            return

        for field in ('due', 'priority'):
            if not task[field] and obj_task[field]:
                task[field] = obj_task[field]

Since it is an add hook, it executes every time a new task is created. What it does is the following: whenever a task is created with a project, its due and priority attributes are set equal to those attributes on the objective task of the project, except if those attributes are explicitly set in the new task. This is very useful since then I can set the priority and due date of a project on the objective task and all tasks of that project will have the same priority and due date automatically.

Demonstration

After talking so much, I owe you at least some gifs showing how this all works out. It's worth saying the following aren't my real tasks, otherwise you'd be looking at more than a hundred on the someday report, for example.

In this first gif, I jump to each report (next, in, standby, objectives and someday) using the g bindings, then add an in task with ai and use S to move it to someday. I also use the <Enter> default binding to inspect the task. You can also see me jumping to tasks using their id and searching for a string using the default binding /.

At asciinema.org

In this second gif, I use the default binding A to annotate the task with simple text and then with the string "Notes". I then use o to make taskopen open a note for the task where I input more annotations.

At asciinema.org

In this last gif, I use the zp binding to show only the stuck projects, then use aN on two of them to create next tasks. Finally, I use gP on a task to show only tasks of its project. Here you can also see the hook in effect, since the Next action 1 task has the same priority and due attributes as the New Project 1 task, even though I didn't specify them.

At asciinema.org

And that's it. If this interested you, take a look at VIT. I only showed my specific bindings and workflow with it, but VIT is capable of a lot more.

Lastly, this post only covered the things related to the managing of tasks and VIT. There are still other very important parts of my organization left to be explained, like how I see my calendar and how I stay on schedule. I'll go over these on the next post.

Porting a flash LED driver upstream

2020-11-20T00:00:00-03:00

Now that I had a working serial cable for my Nexus 5, as described in the "Making an UART cable for the Nexus 5" post, I was ready to face some action and help in upstreaming.

Looking through Brian Masney's TODO page there were a couple options, but the one I ended going for was the rear flashlight. Hardware doesn't get much simpler than an LED and it would be easy to test if it was working.

To my surprise though, the downstream driver had over 3500 lines! But it wasn't only for the flash LED, it supported multiple LED types: WLED, Flash/Torch, RGB, MPP and KPDBL. To make it easier to port and since the flash would be the only one I would be able to test and be sure that worked, I decided to create a new file for the driver and only copy over what was needed for the flash LED.

I started by copying the probe function, compiling the driver, and seeing which errors of missing definitions were thrown. If those definitions had "flash" or "torch" in the name, I would also copy them, otherwise I just removed those references. I repeated this until eventually there were no more missing definition errors and my driver had everything needed for the flash LED.

At this point I still had a lot of compiling errors though, since the downstream driver was for kernel 3.4 and I was compiling for 5.7.6. So I just went through each error, referencing the definitions on both the downstream and upstream kernels, and made the necessary changes.

With the driver successfully compiling, I added a CONFIG for it and enabled it as a module in the defconfig used by Nexus 5 (qcom_defconfig). I also went through the devicetree files on the downstream tree to find which nodes I needed to describe the flash LED hardware to the driver, and the properties needed in them, so I could also add them upstream.

Note: The design of devicetrees can make the properties of a node be scattered across different files. Later I learned that I could compile the downstream kernel and read the source for the complete tree from the blob with dtc -I dtb -O dts -o downstream_dt.dts kernel_lge_hammerhead/arch/arm/boot/msm8974-hammerhead-rev-11.dtb.

With a driver that compiles, a valid devicetree, and configs enabling the driver as a module, I was ready to finally compile my kernel with the driver and flash it to the phone. So I commited my changes and went to battle.

And of course it failed. In fact, it failed so badly that the driver wasn't even binding to the device. Not having a solid understanding of devicetrees and how the binding between the devices and drivers worked, I started researching about it.

A great resource I found was Solving Device Tree Issues (more on eLinux). It was in fact using the dt_node_info script that presentation shows that I found out that the device was being loaded but the driver wasn't. Also, the debugging techniques shown, like enabling the dynamic debug prints on probe functions helped me discover that my driver's probe function wasn't even being called.

After a lot of reading, both online documentation and other drivers' code, I noticed that my driver was registering in the SPMI bus, which made sense to me since it needs to communicate over that bus, but given that the flash LED node in the devicetree was being registered in the platform bus, my driver also needed to, otherwise they would never bind. So that was something I needed to change.

But making my driver register on the platform bus, made me no longer have the spmi_device pointer that I needed to use the SPMI functions to read and write on the registers. Again, looking around on other drivers, like qcom-spmi-iadc, I discovered there was this cool regmap thing I could use instead to read and write on the registers over SPMI but without being specific to SPMI. I did the reasonable thing and just tried it out!

With these changes commited, the driver was now binding to the device, but the probe function was failing with the following messages:

[   14.547394] spmi spmi-0: pmic_arb_wait_for_done: transaction failed (0x3)
[   14.547405] qcom,leds-qpnp fc4cf000.spmi:pm8941@1:qcom,leds@d300: Unable to read from addr=5, rc(-5)
[   14.547503] qcom,leds-qpnp fc4cf000.spmi:pm8941@1:qcom,leds@d300: Regulator get failed(-517)
[   14.547512] qcom,leds-qpnp fc4cf000.spmi:pm8941@1:qcom,leds@d300: Unable to read flash config data

Given that the regulator was the one failing, and that I had just copied over the regulator nodes from the downstream devicetree, the problem was clearly there. I needed to find out what regulators were needed for the LED to work, and add them upstream if they weren't already there.

At this point I emailed Brian Masney asking for some light, and the advice he gave me was to compile and test the downstream kernel. Having something that works to use as a reference, even if it is very outdated, is invaluable.

Using an older toolchain to compile, as instructed in the build script, and after a minor problem, I got the downstream kernel compiling, flashed it, and verified that the LED and the downstream driver indeed worked. I should really have done this from the beginning... Imagine if the flash LED itself was actually faulty!

Then I started exploring the sysfs on this system to see how it worked. I found the regulator that was being used by the LED, whose status went to enabled whenever I turned the LED on with echo 1 > /sys/class/leds/led\:flash_torch/brightness.

With the downstream regulator node and devicetree, and the upstream driver for the regulator (qcom_spmi-regulator) and its dtbinding in hand, I started doing some detective work.

After some investigation I finally discovered that, given that the regulator address was 0xa000, the regulator named 8941_boost downstream is known as s4 in the upstream kernel, or also by its nickname pm8941_5v.

I still had to discover the other regulator's real identity, but having gained some confidence from that fine detective work, and with some tips from the devicetree, I bet all my money that pm8941_chg_boost's real identity was 5vs1, also called pm8941_5vs1.

Commit, flash, test, aaand... nope. It still didn't work, although I had clearly made some progress. Now the probe function was successfully being executed, although the read and write operations on the SPMI registers were still failing:

[   13.346704] leds_qpnp:qpnp_leds_probe: Probe called!
[   13.346746] spmi spmi-0: pmic_arb_wait_for_done: transaction failed (0x3)
[   13.346760] qcom,leds-qpnp fc4cf000.spmi:pm8941@1:qcom,leds@d300: Unable to read from addr=5, rc(-5)
[   13.347250] leds_qpnp:qpnp_dump_regs: ===== led:flash_0 LED register dump start =====
[   13.347285] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x40 = 0x0
[   13.347319] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x41 = 0x0
[   13.347353] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x42 = 0x0
[   13.347385] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x43 = 0x0
[   13.347419] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x44 = 0x0
[   13.347445] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x48 = 0x0
[   13.347470] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x49 = 0x0
[   13.347496] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x4b = 0x0
[   13.347530] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x4c = 0x0
[   13.347563] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x4f = 0x0
[   13.347589] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x46 = 0x0
[   13.347614] leds_qpnp:qpnp_dump_regs: led:flash_0: 0x47 = 0x0
[   13.347622] leds_qpnp:qpnp_dump_regs: ===== led:flash_0 LED register dump end =====

Since I was confident that the devicetree was now right, I went back to the driver code. I sprinkled some pr_debug() over the probe function and noticed that the reg value, which is read from the devicetree and used as the base address for all read and write operations, was 0, even though it should be 0xd300.

Oh. Really?? Well, this wouldn't be a complete adventure without me introducing a dumb bug myself, right? 😝

After fixing the bug, I recompiled, reflashed, rebooted, retested aaand... YESS!!!

Isn't it sooo beautiful? 😍

But let's not get too amazed by that gorgeous light. Now that I finally had a driver that actually worked, I rebased everything on top of mainline (at this point the kernel had already moved from 5.7 to 5.9) to make sure it still worked and sent an RFC patch!

And that's the tale of "How I ported a driver upstream"! Now, that's the end of this post, but not of the adventure. There are lots of things that I need to do to get that driver patch series actually included in mainline (some of which I will still find out from the feedback on my patches).

Since my goal here was always to just to get it working, changing as minimum as possible, and only after that clean it up and make it decent, this is what I'll need to start doing now 🙂.

Just one last thing. This post might have made it seem that the problem resolutions were streamlined, but it wasn't like that at all. There were multiple points were I had no idea what to do, sat on it for multiple weeks, and even thought about giving up and working on another project.

But I'm glad I always kept trying and asking for help, because as much as it was frustrating at times, it was also so much fun and I learned so much. I can't even express my happiness the moment that light finally turned on after 4 months (on and off) of work.

And that's it! I hope to be back in a few months with a new post telling my adventure on getting an RFC patch for a new driver into an actual upstream driver 🙂. See you then!

Bulk file editing with ranger

2020-10-20T00:00:00-03:00

My file manager of choice is ranger. It's terminal-based, provides keybind mapping for everything making me more efficient in navigating my files, and it's incredibly extensible by enabling the creation of custom commands in python. If that wasn't enough, it also has a ton of other great features (extensible file preview, tabs, tags, ...). Go check out their github page, seriously.

One very useful feature of ranger is the bulkrename command. It allows you to open an editor with the filenames of all selected files and edit them. After saving, a shell script is generated to make the necessary renaming (and you're given the chance to review it), and just like magic *poof* you just renamed a bunch of files simultaneously from the convenience of your favorite text editor.

Here's an example:

At asciinema.org

Pretty convenient. But when you think about it, it is rather limited. Why only allow file renaming? This same workflow could be used to edit any file attribute. One just needs to provide a way to obtain the attribute for each selected file and to provide the shell script command that changes the attribute to be what the user changed it to.

The bulk command

I recently wanted to modify the Artist ID3 tag of multiple mp3 files at once which motivated me to write the generic version of ranger's bulkrename command: bulk.

For that I basically copied the code for bulkrename and generalized the file attribute retrieval and the attribute change command generation, by calling the get_attribute() and get_change_attribute_cmd(), respectively, from a bulk subcommand stored in the bulk dictionary.

The bulk command class is as follows:

class bulk(Command):
    """:bulk <attribute>

    This command opens a list with the attribute <attribute> for each of the
    selected files in an external editor.
    After you edit and save the file, it will generate a shell script
    which changes the attributes in bulk according to the changes you did in the
    file.

    This shell script is opened in an editor for you to review.
    After you close it, it will be executed.
    """


    bulk = {}

    def execute(self):  # pylint: disable=too-many-locals,too-many-statements
        import sys
        import tempfile
        from ranger.container.file import File
        from ranger.ext.shell_escape import shell_escape as esc
        py3 = sys.version_info[0] >= 3

        # get bulk command argument
        bkname = self.rest(1)

        # Create and edit the file list
        files = [f for f in self.fm.thistab.get_selection()]
        attributes = [self.bulk[bkname].get_attribute(f) for f in files]
        listfile = tempfile.NamedTemporaryFile(delete=False)
        listpath = listfile.name

        if py3:
            listfile.write("\n".join(attributes).encode("utf-8"))
        else:
            listfile.write("\n".join(attributes))
        listfile.close()
        self.fm.execute_file([File(listpath)], app='editor')
        listfile = open(listpath, 'r')
        new_attributes = listfile.read().split("\n")
        listfile.close()
        os.unlink(listpath)
        if all(a == b for a, b in zip(attributes, new_attributes)):
            self.fm.notify("Nothing to be done!")
            return
        print(new_attributes)

        # Generate script
        cmdfile = tempfile.NamedTemporaryFile()
        script_lines = []
        script_lines.append("# This file will be executed when you close the editor.\n")
        script_lines.append("# Please double-check everything, clear the file to abort.\n")
        script_lines.extend("%s\n" % self.bulk[bkname].get_change_attribute_cmd(file, old, new)
                            for old, new, file in
                            zip(attributes, new_attributes, files) if old != new)
        script_content = "".join(script_lines)
        if py3:
            cmdfile.write(script_content.encode("utf-8"))
        else:
            cmdfile.write(script_content)
        cmdfile.flush()

        # Open the script and let the user review it
        self.fm.execute_file([File(cmdfile.name)], app='editor')
        cmdfile.seek(0)

        # Do the attribute changing
        self.fm.run(['/bin/sh', cmdfile.name], flags='w')
        cmdfile.close()

Then, inside this class, I added a class for each of the bulk subcommands that I wanted to add: id3art, id3tit and id3alb, which change the ID3 tag for the Artist, Title and Album, respectively.

For each one of those, I implemented the get_attribute() and get_change_attribute_cmd() methods. get_attribute() receives a ranger's file object and should return a string with the attribute, in the case of id3art, the ID3 Artist tag, which I used the eyed3 python module to get. get_change_attribute_cmd() receives a ranger's file object, the old attribute (the one returned by get_attribute()) and the new one (the value edited by the user), and should return a string containing the shell script command to apply the change made by the user (in the case of id3art, eyeD3 -a NewArtist fileName.mp3).

Finally, I also added an entry for each of those subcommands to the bulk dictionary, which maps the subcommand name to its object.

Translating all of this to code, this is what I added inside the bulk class:

class id3art(object):

    def get_attribute(self, file):
        import eyed3
        artist = eyed3.load(file.relative_path).tag.artist
        return str(artist) if artist else ''
    
    
    def get_change_attribute_cmd(self, file, old, new):
        from ranger.ext.shell_escape import shell_escape as esc
        return "eyeD3 -a %s %s" % (esc(new), esc(file))

class id3tit(object):

    def get_attribute(self, file):
        import eyed3
        title = eyed3.load(file.relative_path).tag.title
        return str(title) if title else ''
    
    
    def get_change_attribute_cmd(self, file, old, new):
        from ranger.ext.shell_escape import shell_escape as esc
        return "eyeD3 -t %s %s" % (esc(new), esc(file))

class id3alb(object):

    def get_attribute(self, file):
        import eyed3
        album = eyed3.load(file.relative_path).tag.album
        return str(album) if album else ''
    
    
    def get_change_attribute_cmd(self, file, old, new):
        from ranger.ext.shell_escape import shell_escape as esc
        return "eyeD3 -A %s %s" % (esc(new), esc(file))

bulk = {'id3art': id3art(),
        'id3tit': id3tit(),
        'id3alb': id3alb(),
        }

Here it is in action:

At asciinema.org

Since I thought this generic framework for the bulk command could be useful to everyone using ranger, with each user implementing their own bulk subcommand, I opened a pull request. It seems the idea was well received, and they even intend to make bulkrename simply an alias to a bulk subcommand, so maybe in the near future you will be able to make your own bulk subcommands using the built-in bulk command 🙂.

Playlist generation with MPD

2020-09-20T00:00:00-03:00

Music is life. I really love listening to music, although not the same kind of music all the time. Most of the time though, anything goes: I like to listen to any of the songs I have at random. But when I'm doing something that needs concentrating (like writing this text) I can only listen to what I call "background" music, or music that doesn't have vocals. So having groupings of songs is very useful for this kind of situation.

One way of grouping songs is through playlists. But the criteria to determine which songs go into a playlist is very much subjective since, well, it's made up by me, a human being. This is what motivated me to create some way to generate playlists based on varied criteria: folder names, artists, albums, etc.

Recently, I read the incredible book Fluent Python as part of furthering my skills in python, which has easily become my favorite programming language for some time now. I learned a lot reading that book, but hadn't actually practiced any of it yet. One of the most interesting things that I learned was about the special methods that make python objects very flexible. I then realized that using special methods I could make playlist objects that were very flexible while also practicing this magic concept. Win-win.

The program that I use to track my music library and play the music is MPD. It supports querying data about songs and also playing playlists, so it was straight-forward to make my playlist generation script using MPD.

The MPDPlaylist class

This is the idea: I want to be able to create a playlist by specifying what should be in it, and also what shouldn't, and be able to combine criteria from other playlists using AND and OR operations. This will probably become clearer later with the examples.

The code implementing this class is in mpd_playlist.py, which is the following:

from mpd import MPDClient

client = MPDClient()
client.timeout = 60
client.idletimeout = None
client.connect("localhost", 6600)

class MPDPlaylist():

    def __init__(self, query, name=''):
        self.name = name
        if isinstance(query, self.__class__):
            self.songs = set(query.songs)
        elif isinstance(query, set):
            self.songs = set(query)
        else:
            self.songs = self.query_songs(query)


    def __repr__(self):
        return f"{self.__class__}({self.name}, {self.songs})"


    def __or__(self, other):
        return MPDPlaylist(self.songs.union(other.songs))


    def __and__(self, other):
        return MPDPlaylist(self.songs.intersection(other.songs))


    def __neg__(self):
        return MPDPlaylist(MPDPlaylist({}).songs.difference(self.songs))


    def query_songs(self, queries):
        if queries:
            args = []
            for k,v in queries.items():
                args.append(k)
                args.append(v)
            return set(song.get('file') for song in client.search(*args))
        else:
            return set(song.get('file') for song in client.listall() if
                       song.get('file'))


    def write_to_file(self, base):
        if self.name:
            with open(f"{base}/{self.name}.m3u", 'w') as f:
                f.write('\n'.join(self.songs) + '\n')
        else:
            print("Can't write playlist with no name!")

That MPDClient() thing in the beginning is needed to connect to the MPD database to later retrieve all music information. It is provided by the python-mpd2 package.

The __init__() method, which is called when creating a new playlist object, can accept another playlist as its query parameter, in which case the new playlist just copies the songs from the playlist passed. It can also receive a set of songs (which I never actually used, but it made sense to support). And finally, for the most common use, it can receive a dictionary containing the query that will be used to filter the music from MPD. For all cases, an optional name can also be passed for the playlist (needed for the playlist to be saved).

So, for example, paramore_playlist = MPDPlaylist({'artist': 'Paramore'}) would create a playlist only with songs from Paramore, and then paramore_playlist2 = MPDPlaylist(paramore_playlist) could be used to create a copy of that playlist. This second case doesn't sound as useful but is the base of parsing expressions, as we'll see.

__repr__() is just there for debugging. It specifies how the object is printed, showing the songs the playlist contains.

__or__() and __and__() are called when two playlists are OR'ed (using |) and AND'ed (using &) together, yielding a playlist that contains the union (songs from both playlists) and the intersection (only songs that were in both playlists) respectively. __neg__() is for negating the playlist (using -), making so that the query specifies what the playlist should not contain, so it will contain everything but that.

query_songs() uses the query to get all songs that match it from MPD and saves them in the object.

write_to_file() saves the playlist in a .m3u file so that it can be later read and played by MPD. The name of the file is the name given in __init__.

Now, to some examples.

My playlists

In another file, playlists.py, I have all of my playlists defined using the MPDPlaylist class:

from mpd_playlist import MPDPlaylist as PL

saved  = PL({'base': "genres"}, "0_Saved")
buffer = PL({'base': "buffer"}, "1_Buffer")
fvt    = PL({'file': "/%"}, "1_FvtAlbums")

br           = PL({'base': "genres/br"}           , "2_Br")
classical    = PL({'base': "genres/classical"}    , "2_Classical")
electronic   = PL({'base': "genres/electronic"}   , "2_Electronic")
etc          = PL({'base': "genres/etc"}          , "2_ETC")
instrumental = PL({'base': "genres/instrumental"} , "2_Instrumental")
jazz         = PL({'base': "genres/jazz"}         , "2_Jazz")
pop          = PL({'base': "genres/pop"}          , "2_Pop")
post_rock    = PL({'base': "genres/post-rock"}    , "2_Post-rock")
rap          = PL({'base': "genres/rap"}          , "2_Rap")
rock         = PL({'base': "genres/rock"}         , "2_Rock")
soundtrack   = PL({'base': "genres/soundtrack"}   , "2_Sountrack")

common         = PL(rock | electronic | pop | post_rock | rap, "1_Common")
not_soundtrack = PL(-soundtrack, "2_NotSoundtrack")

tdg       = PL({'artist': "Three Days Grace"}, "3_TDG")
minecraft = PL({'base': "genres/soundtrack/games/%minecraft"}, "3_Minecraft")
lotr      = PL({'base': "genres/soundtrack/movies/lotr"}, "3_LOTR")

background = PL(classical
                | jazz
                | instrumental
                | PL({'artist': "Balmorhea"})
                | PL({'artist': "Tycho"})
                | PL({'artist': "MASTER BOOT RECORD"})
                | PL({'artist': "The Album Leaf"})
                | PL({'base': "genres/soundtrack/animes/tatami_galaxy/%ost"})
                | PL({'base': "genres/soundtrack/games"})
                & -PL({'file': "genres/soundtrack/games/Portal - Still Alive.mp3"})
                & -PL({'file': "genres/soundtrack/games/portal_2/3-13_want_you_gone.mp3"})
                & -PL({'title': "Here Comes Vi"}),
                "1_Background")

The first playlists created are:

saved, containing all songs that are in the genres folder, which is essentially my music library;
buffer, with all songs that are inside the buffer folder, which are the ones I'm still listening to decide if I like or not;
fvt with all songs from folders beginning with a %, which are my favorite albums.

Then there are playlists for each of the genres. Some of those, namely rock, electronic, pop, post_rock and rap are then combined with the OR (|) operator to create the common playlist. This means that this playlist contains the music from all of those playlists combined.

Next, the not_soundtrack playlist is created by negating the soundtrack playlist, so the former only contains the music not present in the latter.

The tdg playlist only has music from the Three Days Grace artist.

The last playlist, background, combines several previously defined playlists like classical, and also anonymous playlists like PL({'artist': "Balmorhea"}) (which are just used to create this playlist, and won't be saved as playlists themselves, therefore they aren't saved in a variable and also don't need a name parameter), while also removing specific songs like PL({'file': "genres/soundtrack/games/Portal - Still Alive.mp3"}).

This isn't the most succinct syntax but it also isn't bad and is very flexible: It served all my needs for customizing my playlists.

Saving the playlists

Now, you might be wondering how those playlists are written to the files if they are only created and saved in variables. The answer is: python is cool 😃. This is gen_playlists.py, the cool script that does it:

#!/bin/python
from os.path import expanduser

from mpd_playlist import MPDPlaylist
import playlists

PLD = expanduser("~/.config/mpd/playlists")

# All variables defined locally with type MPDPlaylist and with a name will be
# contained here
playlists = (pl for pl in vars(playlists).values()
             if isinstance(pl, MPDPlaylist) and pl.name)

for playlist in playlists:
    playlist.write_to_file(PLD)

As that nice comment above it says, playlists = (pl for pl in vars(playlists).values() if isinstance(pl, MPDPlaylist) and pl.name) gets all variables of type MPDPlaylist from playlists.py, as long as they have a name. Then the for loop iterates over them and saves each playlist in a file with its name.

Finally, I added a line in cron to run this script every Sunday, updating my playlists.

The 'newest' playlist

Since nothing is perfect, there's one playlist that I couldn't integrate in that framework and was left as its own shell script 🤢: the newest playlist. It contains the 100 latest songs added to my music library.

There are four different timestamps for files according to stat:

time of file birth
time of last access
time of last data modification
time of last status change

To get the latest songs added, and not the ones most recently edited (sometimes I edit song metadata, and don't want it to interfere with this playlist), I needed to use time of birth, but it isn't supported by MPD, so that's why I'm stuck with this gen_playlist_newest.sh shell script:

PLD=$HOME/.config/mpd/playlists
find ~/ark/mus/genres -type f -regextype posix-extended -regex '.*\.(flac|mp3)' -exec stat --format '%W : %n' "{}" \; \
    | sort -nr | head -n 100 | cut -d: -f2 | sed -e 's|.*mus/\(.*\)|\1|' > \
    $PLD/1_Newest.m3u

Automatic context detection for Taskwarrior

2020-08-24T00:00:00-03:00

One of the main ideas of GTD is to have a context associated with each task, so that it is very easy to see which tasks can be done in your current context. I organize my tasks with Taskwarrior, so to make it work with contexts, when adding a new task I need to assign a context to it and also update the current context whenever it changes. There are different types of context, but the easiest to automate are the spatial ones, that is, which tasks I can do where I'm at right now.

Assigning a task to a context is as simple as adding a tag to it. I have three different places I'm normally at, so I either add a @rep, @sp or @uni tag depending on where the task can be done.

To update the current context though I'd need to manually tell Taskwarrior where I'm at right now every time I go to any other place. For example, I'd need to type task context uni every time I went to the university. This is not only very boring but also error-prone: more than once it took me a couple seconds to understand why some tasks were missing.

As any other little problem in life, this can be solved with a little scripting. So that's why I wrote a python script to automatically detect and set the current context for Taskwarrior.

Python script

The idea is very simple: I'm almost always connected to the WiFi, since it connects automatically, and each location has specific WiFi network names, so I just need to get the current WiFi SSID and set the corresponding context.

That is what the following python script does (aside from sending a notification with notify-send):

import subprocess


contexts = {
    'rep': ["rep wifi 1", "rep wifi 2", "rep wifi 3"],
    'sp':  ["sp wifi 1", "sp wifi 2"],
    'uni': ["eduroam"]
}


def get_context(wifi):
    for context in contexts:
        if wifi in contexts[context]:
            return context
    return None


def get_current_context():
    wifi_cmd = subprocess.run(["iwgetid", "-r"], text=True, capture_output=True)
    wifi = wifi_cmd.stdout.split('\n')[0]
    return get_context(wifi)


def set_current_context():
    context = get_current_context()
    if context:
        subprocess.run(["notify-send", "Taskwarrior context",
                       f"Setting context to <b>{context}</b>"])
        subprocess.run(["task", "context", context])
    else:
        subprocess.run(["notify-send", "-u", "critical", "Taskwarrior context",
                       "Failure to detect context"])
        subprocess.run(["task", "context", "none"])

Note: The WiFi names for the rep and sp contexts were redacted to avoid exposure.

Systemd service

Since when I go from one place to the other I always suspend, hibernate or shutdown my notebook, that script only needs to be run after resuming or turning the computer on, which is exactly what the following systemd service is for:

[Unit]
Description=Set taskwarrior context
Wants=network-online.target NetworkManager-wait-online.service
After=network-online.target NetworkManager-wait-online.service hibernate.target suspend.target

[Service]
User=%I
Type=oneshot
Environment=PATH=/usr/bin:/home/nfraprado/ark/code/.path/
Environment=DISPLAY=:0
Environment=XAUTHORITY=%h/.Xauthority
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
ExecStartPre=/usr/bin/sleep 30
ExecStart=/home/nfraprado/ark/code/.path/taskwarrior-update_context
ExecStartPost=

[Install]
WantedBy=multi-user.target
WantedBy=hibernate.target
WantedBy=suspend.target

Details about this service file:

The network-online and wait-online services supposedly make it wait for NetworkManager to connect to a network before executing, but from my tests that wasn't enough, so I ended up adding a 30 second delay as can be seen in ExecStartPre.
The hibernate and suspend targets make it run after the computer resumes or turns on.
The DISPLAY, XAUTHORITY and DBUS_SESSION_BUS_ADDRESS environment variables allow the notification to appear.
taskwarrior-update_context basically calls set_current_context() from the python script.

And that's it! With those two pieces, after I move between two locations and open my notebook the Taskwarrior context is automatically updated, showing me only the tasks relevant to the place I'm at.

Extra: Previous script

As a side note, before that python script, I made a bash script. Even though running a command in bash is way cleaner than python's subprocess.run(), I really despise bash's syntax. I also find it awful to need to define a array_contains function (which I copied from some StackOverflow answer). Anyway, here's the bash script if you're curious:

#!/bin/bash
wifi="$(iwgetid -r)"

declare -a rep=("rep wifi 1" "rep wifi 2" "rep wifi 3")
declare -a sp=("sp wifi 1" "sp wifi 2")
declare -a uni=("eduroam")

array_contains () {
    local array="$1[@]"
    local seeking=$2
    local in=1
    for element in "${!array}"; do
        if [[ $element == $seeking ]]; then
            in=0
            break
        fi
    done
    return $in
}

array_contains rep "$wifi" && context=rep
array_contains sp "$wifi" && context=sp
array_contains uni "$wifi" && context=uni

if [ -z "$context" ];then
    notify-send -u critical "Taskwarrior context" "Failure to detect context"
    task context none >/dev/null 2>&1
else
    notify-send "Taskwarrior context" "Setting context to <b>$context</b>"
    task context $context >/dev/null 2>&1
fi

Setting up mbsync to work with XOAUTH2

2020-07-30T00:00:00-03:00

For a long time I used offlineimap to synchronize my emails between the email providers and my computer. Having access to all my emails offline on my computer is pretty handy. But after seeing the brutal efficiency advantage of mbsync over offlineimap, and having had delay issues with offlineimap myself, I decided to try mbsync out (the project is actually called isync, but the synchronization program is called mbsync).

The main issue I saw with using mbsync is that of the three different emails that I currently use, one of them had to use XOAUTH2 for authentication. This is natively supported by offlineimap but not by mbsync.

Installation

The first step of course was to install isync/mbsync.

I then installed a SASL plugin for the XOAUTH2 mechanism, which in Arch Linux is packaged in AUR as cyrus-sasl-xoauth2-git.

For mbsync to authenticate through XOAUTH2, a program is needed to use the account credentials to obtain the current token. For this I installed a python package called oauth2token which can be installed with pip, or directly from AUR. Thanks to VannTen for creating this package and also for helping me solve some issues with setting up mbsync with XOAUTH2!

Note: When I originally installed mbsync, there was an issue with using it with XOAUTH2. At the time of this writing though it was already fixed in version 1.3.2.

Configuration

To configure oauth2token I followed the instructions in the README, which consist of basically creating two json files with the account information (client_id and client_secret should be setup with the provider, in my case Gmail) in a specific directory, executing oauth2create <provider> <account> and logging into the email account on the browser.

Having set up oauth2token, configuring mbsync to use it was pretty simple. I just added PassCmd "oauth2get <provider> <account>" in the IMAPAccount section of my .mbsyncrc, obviously changing <provider> and <account> for the values I configured in oauth2token. In my case it wasn't necessary to specify the authentication mechanism in that same section with AuthMechs XOAUTH2, since XOAUTH2 is considered the safest mechanism installed, so it is the default. Rather I needed to use AuthMechs PLAIN in the IMAPAccount section of the other accounts that I didn't want to use XOAUTH2 with.

Finally, I ran mbsync for the account that I configured XOAUTH2 with ("dac"). By making the output verbose (-V), it can be seen that it authenticated using XOAUTH2 and everything worked as expected:

[nfraprado@ArchWay ~]$ mbsync -V dac
Reading configuration file /home/nfraprado/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel dac
Opening master store dac-remote...
Resolving imap.gmail.com... ok
Connecting to imap.gmail.com ([2800:3f0:4003:c02::6c]:993)...
Opening slave store dac-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism XOAUTH2...
C: 0/1  B: 0/5  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Maildir notice: no UIDVALIDITY, creating new.
Loading master...
master: 0 messages, 0 recent
Loading slave...
slave: 0 messages, 0 recent
Synchronizing...
C: 0/1  B: 1/5  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0

[ ... ]

Opening master box _Sent...
Opening slave box _Sent...
Creating slave _Sent...
Maildir notice: no UIDVALIDITY, creating new.
Loading master...
Loading slave...
slave: 0 messages, 0 recent
master: 92 messages, 0 recent
Synchronizing...
C: 1/1  B: 5/5  M: +0/0 *0/0 #0/0  S: +367/367 *0/0 #0/0

Making an UART cable for the Nexus 5

2020-06-30T00:00:00-03:00

Recently, me and a friend started digging into making the Nexus 5 run the mainline Linux kernel. The purpose of this is, apart from a great learning experience, to make the Nexus 5 run a Linux distro, like PostmarketOS, instead of Android, while also having lifetime updates delivered from the mainline.

Of course, the first thing before getting to coding is to get the UART from the phone to the computer, so that I could read the log messages from early boot of the kernel and figure out the cause of any issue. In the case of the Nexus 5, the UART is on the audio jack, so a way to connect it to the USB of my computer was needed.

I found instructions about the construction of this cable on both the postmarketOS wiki and on the nexus-5-upstream website, which is a project from Brian Masney, who already contributed a lot of patches to upstreaming the Nexus 5.

The information on these pages were conflicting, so I figured it would be best to prototype the circuit first and make sure it worked before soldering it.

Prototyping

I had to prototype using only the components I already had, since I would already buy some components for the final board, and didn't want to do two purchases.

I started following the schematic on the postmarketOS wiki, since it was based on a schematic from Google.

For the audio jack I used an old earphone I had around, and cut it so that I ended up with only the cable with the wires exposed and the connector. I followed the same resistor values on the schematic, using a 1kΩ and two 100Ω resistors in series to get the 1.2kΩ for the voltage divider on the TX pin.

To bridge UART to USB I used the Adafruit FT232H breakout I had. Since it didn't have a 3.3V output, I tried some combination of resistors to divide the 5V. That got me to 3.26V but it still didn't work.

I was almost giving up, but remembered that Brian Masney wrote in that page that it really needs to be at 3.3V, and I also remembered I had an SPI to USB converter board that had a 3.3V regulated output. I decided to try it out.

And after only a minor hiccup, with having to change the baud rate to 115200, what do you know, it worked! 🥳

This is how the prototype looked:

I know, it looks horrible, but hey, it works!

Having got it to work I ordered some more parts to make the circuit neater.

Final board

The components I used to build the board were:

wires
1 universal PCB (at least 9x7 holes)
1 1kΩ resistor
1 1.2kΩ resistor
1 1x7 male pin header
1 TRRS audio jack cable (from an old earphone) Note: it really needs to be a TRRS, that is, to have 4 separate conductors.
1 UART <-> USB converter (from a store in Brazil, but any with 3.3V is fine)

For reference, this is how the pins should be connected (but you really should refer to the schematic at postmarketOS):

UART pin	Between	Audio jack pin
RX	—	Tip (TX)
TX (at 3.3V)	Voltage divider (1kΩ and 1.2kΩ)	Ring 1 (RX at 1.8V)
GND	—	Ring 2
3.3V	—	Sleeve

I invested a little time thinking how was the most neat way I could solder the connections and resistors to the board, and I ended up with this:

And with the connections on the back side and corresponding pins shown:

My PCB was bigger than the converter board so I sawed it to have approximately the same size (17x7), but the connections only take up a 9x7.

I also changed the jumper position on the UART to USB converter board so that VCC was 3.3V.

Finally I soldered the male pin headers to the converter board, and that on top of my PCB, ending up with a pretty cool 2-story board 😎.

This is how the final board looks:

Testing

After connecting the board's USB to my computer and the audio jack to the Nexus 5, I opened a serial console using picocom with a 115200 baud rate:

picocom /dev/ttyUSB0 -b 115200

And booted the Nexus 5 into fastboot mode by holding the power and volume down buttons, to be greeted with this heartwarming message:

welcome to hammerhead bootloader
[10] Power on reason 80
[10] DDR: hynix
[110] Loaded IMGDATA at 0x11000000
[110] Display Init: Start
[190] MDP GDSC already enabled
[190] bpp 24
[230] Config MIPI_CMD_PANEL.
[230] display panel: ORISE
[230] display panel: Default setting
[360] Turn on MIPI_CMD_PANEL.
[410] Display Init: Done
[410] cable type from shared memory: 8
[410] vibe
[610] USB init ept @ 0xf96b000
[630] secured device: 1
[630] fastboot_init()
[680] splash: fastboot_op
 FASTBOOT MODE
 PRODUCT_NAME - hammerhead
 VARIANT - hammerhead D821(H) 16GB
 HW VERSION - rev_11
 BOOTLOADER VERSION - HHZ20h
 BASEBAND VERSION - M8974A-2.0.50.2.30
 CARRIER INFO - None
 SERIAL NUMBER - ***
 SIGNING - production
 SECURE BOOT - enabled
 LOCK STATE - unlocked
[790] splash: start
[1820] Fastboot mode started
[1820] udc_start()

Since I verified that the pinout on nexus-5-upstream was wrong, I sent a pull request fixing it, so there's no such confusion anymore 😉.

Great! Now that I can read all boot logs I'm ready to dive into the kernel code. Although I'm sure this was the easiest part of the project 😅...

Creating movie and game lists using Taskwarrior

2020-05-25T00:00:00-03:00

I like watching movies and playing games, but I also like to keep track of which movies I've watched and which games I've beaten. I also got into the habit of rating each movie I watch. For the longest time, though, I have kept track of these collections informally in my memory or scattered in text files on my computer.

A couple years ago I started trying to keep myself more focused and organized by using a CLI (Command-line interface) TODO list manager called Taskwarrior. Suddenly I realized that I could also use this awesome tool to keep lists for my movies and games!

Taskwarrior configuration

If you know Taskwarrior, you know that it just needs a configuration file (normally a .taskrc in your home folder), and a folder to keep its data files.

In order to keep these lists separate from my main Taskwarrior tasks, I created a directory (/home/nfraprado/txt/todo) to keep a separate configuration file (config) and data folder (data).

Inside config, I first defined the data folder:

data.location = /home/nfraprado/txt/todo/data

I then removed color from tagged tasks and made search case insensitive:

color.tagged = none

search.case.sensitive = no

And also created a new UDA, or User Defined Attribute, called Score, so that I could add a score to a movie after I watched it:

uda.score.type = numeric
uda.score.label = Score

Finally, I added the movies and games reports. Each report can have its filter configured to determine which tasks are shown, and its columns, to determine which task attributes are displayed, as well as some other configurations.

For the games, I wanted two reports: one called games.todo, to show which games I still need to beat, and another called games.done for the games I've already beaten:

# Games reports
report.games.todo.description = Games to complete
report.games.todo.labels = ID,Title
report.games.todo.columns = id,description
report.games.todo.filter = status:pending limit:page project:games

report.games.done.description = Games completed
report.games.done.labels = Title,Completed on
report.games.done.columns = description,end
report.games.done.filter = status:completed limit:page project:games
report.games.done.sort = end-

For the movies, I also added a todo and a done report, but the done report also has a score column so that I can see the score that I gave to each movie. I also added a movies.rank report for the movies that is sorted based on the score, so I can have a list of my favorite movies 🙂:

# Movies reports
report.movies.todo.description = Movies to watch
report.movies.todo.labels = ID,Tags,Title
report.movies.todo.columns = id,tags,description
report.movies.todo.filter = status:pending limit:page project:movies

report.movies.done.description = Movies seen
report.movies.done.labels = Title,Tags,Score,Watched on
report.movies.done.columns = description,tags,score,end
report.movies.done.filter = status:completed limit:page project:movies
report.movies.done.sort = end-

report.movies.rank.description = Movies ranking
report.movies.rank.labels = Title,Tags,Score
report.movies.rank.columns = description,tags,score
report.movies.rank.filter = status:completed limit:page project:movies
report.movies.rank.sort = score-

Bash configuration

With Taskwarrior configured, I just needed some aliases in my .bashrc to make it convenient to add, complete and see the movies and games.

Inside my .bashrc, I first added a variable pointing to the config location:

TODOCONFIG="$HOME/txt/todo/config"

Then I added aliases to add, list and list done for both games and movies. Also an alias to mark a game as completed and one to see the movies ranking. For all these commands, I used rc:$TODOCONFIG to make Taskwarrior use the configuration file that I previously added. For the add and done aliases I just use the add and done Taskwarrior commands, respectively. Listing is done by just calling the previously written reports:

# Games list
alias gamadd='task rc:$TODOCONFIG add project:games'
alias gamlist='task rc:$TODOCONFIG games.todo'
alias gamlistdone='task rc:$TODOCONFIG games.done'
alias gamdone='task rc:$TODOCONFIG done'

# Movies list
alias movadd='task rc:$TODOCONFIG add project:movies'
alias movlist='task rc:$TODOCONFIG movies.todo'
alias movlistdone='task rc:$TODOCONFIG movies.done'
alias movrank='task rc:$TODOCONFIG movies.rank'

Completing a movie can't be done as a simple alias, since I also want to give it a score, so I create a simple bash function that takes the task id and score, applies the given score to the task and marks it as done:

movdone()
{
    if [ -z "$1" ] || [ -z "$2" ];then
        echo "Usage: movdone id score"
        return 1
    fi
    task rc:$TODOCONFIG modify "$1" score:"$2"
    task rc:$TODOCONFIG done "$1"
}

Result

With this setup done, I could finally track my movies and games very easily.

Adding a new movie to watch is as simples as movadd movie_name (spaces can actually be used in the name).

And after watching a movie, I can easily mark it as completed and rate it with movdone id rating. The id for the movie can be seen using movlist.

Here are the movies that I plan to watch:

[nfraprado@ArchWay ~]$ movlist
Using alternate .taskrc file /home/nfraprado/txt/todo/config

ID Title
23 Once Upon a Time In Hollywood
27 Ready player one
28 In The Shadow Of The Moon 2007

The last 5 movies I've watched:

[nfraprado@ArchWay ~]$ movlistdone
Using alternate .taskrc file /home/nfraprado/txt/todo/config

Title                                             Tags   Score Watched on
Arrival                                                      8 2020-05-25
The Green Mile                                               8 2020-05-17
The Boy Who Harnessed the Wind                               8 2020-04-25
Jojo Rabbit                                                  6 2020-04-18
Back to the Future                                           8 2020-04-12

My highest rated movies (since I created this list):

[nfraprado@ArchWay ~]$ movrank
Using alternate .taskrc file /home/nfraprado/txt/todo/config

Title                                             Tags   Score
The Lord of the Rings: The Return of the King               10
The Lord of the Rings: The Two Towers                       10
The Lord of the Rings: The Fellowship of the Ring           10
Whisper of the Heart                              ghibli     9
Apollo 11                                         doc        9
Life is beautiful                                            9
Koe no katachi                                               9

Some games I plan to beat:

[nfraprado@ArchWay ~]$ gamlist
Using alternate .taskrc file /home/nfraprado/txt/todo/config

ID Title
 6 FTL
 7 The Witness
 8 Factorio
 9 Stardew Valley
10 Half Life

And the last 5 games I've beaten:

[nfraprado@ArchWay ~]$ gamlistdone
Using alternate .taskrc file /home/nfraprado/txt/todo/config

Title                                                      Completed on
The Stanley Parable                                        2020-04-24
Undertale                                                  2020-04-17
Cave Story                                                 2020-03-30
Antichamber                                                2020-03-22
Papers, Please                                             2020-03-19