A while ago Erwan Leroy, a VFX professional and trainer, contacted me with some questions regarding the catalogs in Gaia Sky. Basically, he was trying to decode the binary format used in Gaia Sky to load the stars using a Python script. Of course, my documentation was lacking in that very aspect, so I walked him through the format and then improved the docs.

Today, he has come back to me to share his results. Turns out he was trying to replicate an astrophoto with rotation star trails using his visual effects skills. And he came pretty damn close, I might add. I read the article and liked it, so I am sharing it here: Rendering a plausible night sky.

In my re-implementation of the Gaia Sky level-of-detail (LOD) catalog generation in Rust I have been able to roughly halve the processing time, and, even though I do not have concrete numbers yet, everything points towards a drastic decrease in memory usage as well. In this project, I need to read a metric ton of gzipped csv Gaia catalog files, parse and process them into a functional in-memory catalog with cartesian positions, velocity vectors, RGB colors, etc. Then I need use them to generate an octree that represents the LOD structure, and finally write another metric ton of binary files back to disk. Using memory mapped files helps a lot in avoiding copies and speeding up the reading and writing operations; that’s something I tried out in the Java version and have come to also re-implement in Rust. Here’s the thing though: working with memory mapped files in Java is super straightforward. In Rust? Not so much. And the lack of available documentation and examples does not help. I was actually unable to find any working snippets with all the parts I needed, so I’m documenting it in this post in case someone else is in the same situation I was.

A little over a year ago, in January 2020, I got myself a QNAP TS-351-2G 3-bay NAS in order to store all of my and my family’s data in a fail safe RAID configuration. I opted for the somewhat unconventional 3-bay setup in an attempt to trade off limited physical space at home with storage capacity. I don’t have much space in my living room for a big NAS, and the 2-bay options, albeit being very compact, are limited to RAID-1, where half of the space is used for storage and the other half is used for redundancy protection (data is basically mirrored on the second drive). In QNAP’s website there are three 3-bay Home options: the entry-level TS-332X, the middle-range TS-328 and the high-end TS-351. So I thought to myself, “I’m getting the high-end unit, how bad can it be?”. Well, now that I have been using this NAS for a year I think I can answer this question.

The other day I was browsing through the Wayback Machine and I tried looking up if they had a copy of my old blog dimlight.wordpress.com, a blog I wrote during my second Erasmus exchange program stay in the University of Aberdeen, Scotland. Of course, they had several snapshots archived. The images and videos are missing, but the texts are intact, so I decided to re-publish verbatim copies of these old posts here.

If you usually develop your software without an IDE, it may come in handy to be able to run a custom command or two whenever a file or a group of files in the file system is modified. This post discusses entr, a small event notify test runner which might just be what you need to fill an inconvenient gap in your mouseless development environment.

Lately I’ve taken an interest in the Rust programming language. I read the epub version of the Rust book over summer and found it quite well-written and overall interesting. Recently, I got myself a hard copy that sits on my desktop for quick queries, as I’ve been re-implementing the Gaia Sky catalog generation in Rust. The reasons for this are varied, but basically they come down to hardware. We need to load a humongous amount of stars (~1.6 billion) into memory, our old server, which had 2 TB of RAM, is being replaced by a newer, faster, and smaller (memory-wise, ~700 GB) machine in which we won’t be able to generate with the current Java implementation. But this is a story for another post. Today I’m here to document the completely mouseless Rust development environment I’ve been putting together as a sort of side project, to aid in my re-implementation.