Projects in progress: June 2022

Macoy Madson

This article is mirrored on my blog.

I have several projects written in Cakelisp that I have been chipping away at.

File Helper

I haven't made much progress on File Helper, but I do find myself reaching for it and hope to revisit it.

I want to create a command-line interface for creating an index of files on systems I have which aren't graphical. I can then tag and visualize the file system on my work station.

Machsearch

My newest application does indexing full-text search. It's much faster than ag, grep, or any other non-indexing full-text search.

It's so fast that you can search as you type, and it scales very well to large codebases thanks to its index.

I implemented a self-update feature for Machsearch which I think will come in handy on many of my previous and future applications.

There isn't much competition in the indexed file search space, but there are three main products out there:

  • Google Zoekt: Google originally made Codesearch, which is what powers Machsearch. Now, they seem to have moved to this new platform which is a client-server model rather than a CLI application.
  • Sourcegraph is a company whose chief product is a code navigation suite. It is at least in part built on Zoekt, judging by their open-source fork of it.
  • Microsoft Visual Studio 2022 is adding an indexed file search.

I think there is still room for Machsearch in this environment, for three reasons:

  • Machsearch is a single executable. There's no server or heavyweight IDE needed. There are fewer things which can go wrong with this.
  • No project file is necessary: the only required step is selecting which directories you want to index.
  • Machsearch's Custom Commands feature makes it trivial to follow up with additional processing using other command-line tools. The other offerings are either not running on your machine or editor-centric.

Distributed automation

My most recent project has been a continuous integration system, distributed-automation. I am building it to fit my needs exactly, and do not expect it to be of interest to others.

Workers on various different machines communicate with a master server and do work.

I decided to build this rather than "buy" existing CI products like Jenkins or BuildBot for two reasons:

  • These systems have relatively complex configurations. I tried to set up BuildBot, but their getting started tutorial almost immediately failed with no error and no logs to help me debug it. If the "happy path" doesn't even work, I don't feel confident exploring further.
  • I haven't done any programming with networking or sockets, so this automation system gives me valuable experience with this kind of API.

I will take more time setting this up, but in the end I will have a perfect fit for my needs (and no more!) as well as gained knowledge with network programming.

I plan on using this system to automate all of my builds, especially multi-platform builds and releases. This will make it easier for me to create binary releases of all of my projects. It will also make it easier for me to detect platform compatibility and regressions for Cakelisp and GameLib.

While somewhat unusual for a CI, I am planning on using it for automating my backups as well.

Linker/loader

My Linker-loader project has been slow going. I have somewhat de-prioritized it because of its heavily R&D nature, but I do want to continue making gradual progress on it. Even if I don't end up using it, I have had to learn much more about the linking and loading stage, which is both demystifying and enabling for more advanced runtime code loading I'm sure I'll try in the future.

There is a significant amount of incidental complexity and cruft involved with linking. It can be pretty uninteresting and frustrating to deal with.

Cakelisp

Cakelisp has been very stable. I had to make a few changes to fix relative paths recently, but that's about it. I'm very happy with how little I need to work on the language.

I have been considering building a basic IDE for Cakelisp, similar to what Beef did. I would do this primarily to ease the transition from "I'm on the home page of Cakelisp and want to try it" to "I wrote/modified and executed Cakelisp successfully".

Thanks to work on File Helper and Machsearch, as well as some prototypes I have made, this IDE would not take much more than a few weeks to put together, and would substantially lower the barrier to entry.

GameLib

GameLib is my collection of modules for various 3rd-party dependencies as well as useful pure-Cakelisp utilities. Here is a list of some relatively new additions:

  • Network.cake for basic socket-based networking on GNU/Linux and Windows
  • AutoUpdateApplication.cake for handling application self-updating
  • VersionedData.cake for data serialization versioning. This is very minimal for now, but enough to handle my existing application user configuration demands.
  • XML.cake, which cleanly generates XML.
  • Various 3rd-party libraries: Raylib, curl, miniz, Oniguruma, and FreeType. These integrations allow for easy importing of these libraries without requiring a wrapper that would quickly go out of date.

Summing up

As you can see, I have many different projects I'm working on, all happily written in Cakelisp. The projects drive the language's development, and the language's ecosystem continues to grow. It's still just me using Cakelisp for now, but I'm okay with that.

Is there a build of Machsearch that I could try ?

Sure, email me at [email protected] with your preferred platform and I'll send you a build.