This post is also on my blog.
I have been working hard on a file organization application written in Cakelisp. This is part of my career goal to sell my software. The project's name is File Helper (I haven't spent any time trying to come up with a better name yet).
I believe I have a good approach to thinking about my projects. It is sometimes hard to define project success in terms other than "other people think it's really cool". There are a large number of reasons why a project is still a success, even if it ends up not being popular.
Accumulating code
I have been continuously improving Cakelisp and GameLib and intend to continue using these tools on future projects. This means all the bugs I find and fix, the features I add, and the body of references I accumulate all contribute to my potential for future success.
The more I can leverage and improve my existing tools, the easier it will be to create better applications. I started Cakelisp because I believed it would help me make better software faster, and I would be happier working with it. I still believe I have achieved these results.
Accumulating experience
Before File Helper, I hadn't had much experience writing shaders for 3D graphics. My career has always been focused on gameplay software engineering. File Helper's Tree Map gave me an opportunity to write some simple shaders and get my feet wet with OpenGL. I am already excited to use this knowledge on my next project, which will definitely need some shaders, because it also requires some custom graphical widgets.
Additionally, the more projects I get to version 1, the better I get at planning and executing my plans. I feel I have gotten better at knowing when to set artificial deadlines for myself, and how to more effectively filter the features which waste time vs. the ones that really matter.
Determining what's worth making
It is very difficult to know whether something is worthwhile until you have something to put in front of people. This is partly the idea behind the Lean startup. I knew the reasons I wanted File Helper, but I do not know how many other people would want it. Do others encounter similar problems to what made me want the tool?
I needed to create something in order to answer this question. I plan on having an MVP for File Helper online and available to download in October. This project was started around 5 months ago, and developed in earnest (nearly every day) for around 2 months. I would like to shorten this time for the next project, especially by gathering input from potential users much earlier, which is something I didn't do with File Helper.
If File Helper doesn't find an audience, I gained an extremely valuable data point. I'd know I am making the wrong thing[^1], and I would hopefully also receive user feedback to hint me in a better direction. I think spending a couple months to learn this lesson isn't entirely unreasonable, especially because I still gain a tool which is valuable to me at the end of it.
Crafting things is satisfying
I'm very proud of the projects I have finished. It feels good to create things, and even if those things "fail", the journey of putting them together, the "Aha!" moments, and tinkering with the whirring machine you built is still worth a ton.
[^1]: In the context where my end goal is to make enough money on my projects to sustain myself, which is my career goal.
Cakelisp Alpha 3 is now released on GitHub.
It has been a while since I last pushed a release. This is solely because I didn't have any set milestone I was targeting for v0.3.0. Nevertheless, I got a lot done:
Overall, the language is in a really good state. I have been working solidly on several projects (some completed) written in Cakelisp:
I think the robustness of the language is proven by how productive I have been without having to make any large changes to Cakelisp itself. An important note is that I created Cakelisp as a means to an end, not the end itself. Cakelisp changes are driven by my personal projects requiring them, which I think proves feature usefulness unlike toy language features. I am getting things done using Cakelisp, and I believe I get things done faster and better than without it.
If you're interested in seeing the awesome stuff I'm learning and creating with Cakelisp, check out my website and Handmade Network page. As always, please feel free to email me at [email protected] with any comments, feedback, or questions.
It has been a while since I last pushed a release. This is solely because I didn't have any set milestone I was targeting for v0.3.0. Nevertheless, I got a lot done:
- Compile-time code uses a precompiled "CakelispComptime" header. This speeds up compile-time code compilation by about 50% in my tests
- `tokenize-push` was overhauled to be in a really good place going forwards. This was important to allow macros with unlimited token array sizes
- Types now support arbitrary keywords, which means `volatile`, `unsigned`, etc. are supported instead of just `const`
- Various C and C++ constructs are now included in `runtime/` such as `defenum` for `enum` support and a work-in-progress `class` generator
- Tokenizer is more robust to weird edge-cases
- Emacs support is improved
- Many bug fixes
Overall, the language is in a really good state. I have been working solidly on several projects (some completed) written in Cakelisp:
- GameLib, which serves as a great reference for various powerful features enabled by Cakelisp such as introspection, auto-instrumentation of functions for profiling (inspired by Jonathan Blow's Code Modification demo), and more
- Kitty Gridlock, an Android game written in Cakelisp
- File Helper, a treemap file organization application
I think the robustness of the language is proven by how productive I have been without having to make any large changes to Cakelisp itself. An important note is that I created Cakelisp as a means to an end, not the end itself. Cakelisp changes are driven by my personal projects requiring them, which I think proves feature usefulness unlike toy language features. I am getting things done using Cakelisp, and I believe I get things done faster and better than without it.
If you're interested in seeing the awesome stuff I'm learning and creating with Cakelisp, check out my website and Handmade Network page. As always, please feel free to email me at [email protected] with any comments, feedback, or questions.
The following post is mirrored on my blog.
I have been making progress on several fronts with Cakelisp and GameLib, but in this post I wanted to focus on an exciting work-in-progress addition I have: type introspection.
What it is
Type introspection refers to the ability to inspect types at runtime. Reflection is the ability to both inspect and modify types at runtime, which I'm not going to talk about here. By types, I'm referring primarily to structured data, e.g. struct in C.
What this means is that given a type my-struct, you could write a loop which iterates over the fields in that struct at runtime and know some information about that field, without having to write a single line of code per each field.
I am going to attempt to convince you why this is an essential feature to have in your arsenal.
Applications
As soon as you can programmatically iterate over the fields of a type, many doors are opened.
Serialization
The first application for type introspection is perhaps the most obvious. The type metadata is used at runtime to serialize the struct to and from various formats. This eliminates a huge amount of error-prone, repetitive, and just plain boring-to-write serialization code.
For example, with introspection support I can write the following:
This writes the following text to stderr (but could just as easily write to a file):
I chose a Lisp-style format for the plain text, but XML, JSON, etc. could easily be generated instead.
Plain-text is a great initial serialization format because you can easily read and edit it with a regular text editor. This means you now have data or configuration files. All you need to do is def-introspect-struct the configuration struct, then e.g.:
Schemas
Once you have metadata for a type, there's no reason why you can't use that same functionality on the metadata itself. If you do this, you can then serialize your metadata and export it to other programs. This is essential when writing editors or other programs which cannot have knowledge of the original type (because it is in a different programming language, for example). You could also generate documentation for the types, automatically create a suitable SQL table to store data of the type, or generate e.g. a REST API specification.
Debugging and hot-reloading
Reading and writing structs at runtime is a valuable debugging tool as well. You can print structs to stderr to inspect or log their values, which is much easier than writing a printf statement for structs with many fields.
You could write runtime structs to a file, then watch that file for changes, and hot-reload the file with the new struct values!
Taken a step further, you could provide a custom interface for viewing and editing runtime data and configuration. I've seen this in action at a previous job, where their MMO shard consisted of a dozen or more different servers. They provided the interface over HTTP, so you could access any shard's server via URL in any web browser. When things went wrong you could inspect and even modify values in the web browser, then see the change in-game without having to restart anything. Isn't that incredible?
Editors
On any major game project in development, there comes the problem of how people without knowledge of programming input data into the game (many artists and designers, for example). The solution is commonly to create custom editors for the data. This creates a large amount of work for game programmers because these editors must be created in-house.
Off-the-shelf editors do not have enough context to properly serve AAA game developers' needs. For example, a drop-down field which provides all possible values of an enumeration greatly reduces data entry errors and increases productivity, but no off-the-shelf editor knows the values of your enumerations without being told.
So, we are stuck with implementing our own editors. Compare this pseudo-code:
...to this:
Which would you rather spend your precious life time typing? The straight-line version may be simple for a few editors, but quickly becomes more complex once you must maintain dozens of editors, all of which could have several bugs.
The introspection-powered version not only saves you typing every time a field is added, renamed or removed. It also reduces bugs by centralizing all the complexity to one function rather than distributing it to each editor.
If your metadata format supports tags, you can add a tag like no-edit to hide a field. You can also decide to have create-editor check fields against a table of override functions, then defer creation of editor widgets to an override. For example, a color picker widget could be selected for ([] 3 float) rather than displaying three unintuitive sliders.
An added benefit is that the code which edits the structure is completely separated from knowing the actual type of the structure. This means you could write a generic editor which can understand structs it was not compiled to understand, simply by passing the metadata for that struct. You could even write the editor in another language, if you so choose.
You can see an example of introspection for editors taken to an extreme in Unreal Engine. New structs/classes declared with UCLASS or USTRUCT prefixes automatically create editor interfaces through type introspection and UPROPERTY annotations.
Remote procedure calls
Once you have a system for easily serializing types, creating a remote-procedure call system is much easier. When an RPC is initiated, the system serializes all the arguments into a packet along with the procedure name to call. On the other side of the connection, the arguments are deserialized, the procedure is found from a function table, and then it is called.
You will likely want to use code generation for doing the argument unwrapping and forwarding to the procedure.
How it can be implemented
Some languages, like Python, provide type introspection as a built-in feature of the language. C, for example, does not provide type introspection. There are several approaches you can use to add type introspection to any language:
- Code parsing: In a stage before compiling your native language files, a separate executable parses the native language files for type definitions and generates type metadata. This generated metadata can either be stored in a custom format and loaded at runtime, or output to a new native language file (code generation) and compiled with the rest of the files. This technique is used by Unreal Engine to add various features on top of C++[0], for example
- Compiler modification: A modified version of the language's compiler could be created that understands your custom annotations and generates the type metadata. The disadvantage is complexity (maintaining a compiler) as well as portability[1]
- Separate schema: Rather than defining the types in the native language (e.g. C), define schemas in a format that is easy to parse. You then use a code generator to generate the native language type definitions as well as the metadata for the type. This is the approach used by Naughty Dog (convention link) and Google's FlatBuffers, for example. The disadvantage of this approach is the disconnect between the actual implementation - you now have a "special language" that you must write in rather than what you (and your teammates) are familiar with
The big advantage of implementing type introspection on your own is that you can customize it to your needs, whether they be specialized type declarations, different performance characteristics, et cetera.
How introspection is implemented in Cakelisp
Cakelisp has both full-power macros and arbitrary compile-time code generation and scanning, which make type introspection implementable in user-space. That is to say, it is possible to implement type introspection without having any prior support for it in Cakelisp itself.
My work-in-progress implementation of introspection is in GameLib. I added introspection to Cakelisp by creating a macro def-introspect-struct which follows regular defstruct syntax, but allows optional tags to be added to customize the metadata. Metadata is code-generated to the file by the same macro so it can be easily used at runtime. The generated C++ for my-struct, defined previously, looks like this:
This data is never edited by hand, and automatically updates whenever the definition to my-struct updates. I will add more types, flags, and fields to the metadata format as necessary.
Conclusion
Type introspection is a great way to:
Next time you are copy-pasting code or dealing with boilerplate after adding a new field to a structure, consider whether type introspection could be applicable.
[0] Parsing C++ is extremely complicated, so you can decide either to support a subset of C++ when defining types or bring in a beastly dependency like libclang to help. A primary factor of my starting Cakelisp was how frustrated I was with C++'s parsing complexity. The syntax effectively encrypts the type and function signatures I need to do things like introspection.
[1] Some platforms, especially game consoles, require you to use their compiler, which they themselves modified, and likely will not release source code of.
I have been making progress on several fronts with Cakelisp and GameLib, but in this post I wanted to focus on an exciting work-in-progress addition I have: type introspection.
What it is
Type introspection refers to the ability to inspect types at runtime. Reflection is the ability to both inspect and modify types at runtime, which I'm not going to talk about here. By types, I'm referring primarily to structured data, e.g. struct in C.
What this means is that given a type my-struct, you could write a loop which iterates over the fields in that struct at runtime and know some information about that field, without having to write a single line of code per each field.
I am going to attempt to convince you why this is an essential feature to have in your arsenal.
Applications
As soon as you can programmatically iterate over the fields of a type, many doors are opened.
Serialization
The first application for type introspection is perhaps the most obvious. The type metadata is used at runtime to serialize the struct to and from various formats. This eliminates a huge amount of error-prone, repetitive, and just plain boring-to-write serialization code.
For example, with introspection support I can write the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | ;; Define a struct which is introspectable
(def-introspect-struct my-struct
name (* (const char))
value int
decimal float
bad ([] 3 float) (ignore) ;; Don't try to introspect this field - ignore it
truthy bool
charry char)
;; Create an instance of the struct, for testing
(var a my-struct (array "Test struct" 42 -0.33f (array 1.f 2.f 3.f) false 'a'))
;; Write out the struct, passing in the metadata for introspection
(write-introspect-struct-plaintext (addr my-struct--metadata) (addr a) stderr)
|
This writes the following text to stderr (but could just as easily write to a file):
1 | (my-struct :name "Test struct" :value 42 :decimal -0.330000 :truthy false :charry 0) |
I chose a Lisp-style format for the plain text, but XML, JSON, etc. could easily be generated instead.
Plain-text is a great initial serialization format because you can easily read and edit it with a regular text editor. This means you now have data or configuration files. All you need to do is def-introspect-struct the configuration struct, then e.g.:
1 2 3 | (var configuration my-config (array 0))
(read-introspect-struct-plaintext (addr my-config--metadata) (addr configuration)
(fopen "config.cakedata" "r"))
|
Schemas
Once you have metadata for a type, there's no reason why you can't use that same functionality on the metadata itself. If you do this, you can then serialize your metadata and export it to other programs. This is essential when writing editors or other programs which cannot have knowledge of the original type (because it is in a different programming language, for example). You could also generate documentation for the types, automatically create a suitable SQL table to store data of the type, or generate e.g. a REST API specification.
Debugging and hot-reloading
Reading and writing structs at runtime is a valuable debugging tool as well. You can print structs to stderr to inspect or log their values, which is much easier than writing a printf statement for structs with many fields.
You could write runtime structs to a file, then watch that file for changes, and hot-reload the file with the new struct values!
Taken a step further, you could provide a custom interface for viewing and editing runtime data and configuration. I've seen this in action at a previous job, where their MMO shard consisted of a dozen or more different servers. They provided the interface over HTTP, so you could access any shard's server via URL in any web browser. When things went wrong you could inspect and even modify values in the web browser, then see the change in-game without having to restart anything. Isn't that incredible?
Editors
On any major game project in development, there comes the problem of how people without knowledge of programming input data into the game (many artists and designers, for example). The solution is commonly to create custom editors for the data. This creates a large amount of work for game programmers because these editors must be created in-house.
Off-the-shelf editors do not have enough context to properly serve AAA game developers' needs. For example, a drop-down field which provides all possible values of an enumeration greatly reduces data entry errors and increases productivity, but no off-the-shelf editor knows the values of your enumerations without being told.
So, we are stuck with implementing our own editors. Compare this pseudo-code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | (defun power-definition-editor (the-power power-definition) (create-text-field "Name: " (field the-power name)) (create-text-field "Description: " (field the-power description)) (create-float-slider "Damage: " (field the-power damage) -100.f 100.f) ;; etc... ) (defun enemy-definition-editor (the-enemy enemy-definition) (create-text-field "Name: " (field the-enemy name)) (create-text-field "Model Name: " (field the-enemy model-name)) (create-float-slider "Health: " (field the-enemy health) 0.f 100000.f) ;; etc... ) ;; Continues, for every data definition you ever want to have an editor for |
...to this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | (defun create-generic-editor (metadata type-metadata editing-struct (* void))
(for-each-field-in-metadata metadata current-field
(when (= field-type-string (field current-field type))
(create-text-field (field current-field name)
(+ editing-struct (field current-field offset))))
(when (= field-type-float (field current-field type))
(create-float-slider (field current-field name)
(+ editing-struct (field current-field offset))))))
;; Then:
(create-generic-editor power-definition-metadata the-power)
(create-generic-editor enemy-definition-metadata the-enemy)
;; We can even load the metadata at runtime and create the editor based on that:
(create-generic-editor (read-introspect-struct-plaintext "TheWorld.metadata") the-world))
|
Which would you rather spend your precious life time typing? The straight-line version may be simple for a few editors, but quickly becomes more complex once you must maintain dozens of editors, all of which could have several bugs.
The introspection-powered version not only saves you typing every time a field is added, renamed or removed. It also reduces bugs by centralizing all the complexity to one function rather than distributing it to each editor.
If your metadata format supports tags, you can add a tag like no-edit to hide a field. You can also decide to have create-editor check fields against a table of override functions, then defer creation of editor widgets to an override. For example, a color picker widget could be selected for ([] 3 float) rather than displaying three unintuitive sliders.
An added benefit is that the code which edits the structure is completely separated from knowing the actual type of the structure. This means you could write a generic editor which can understand structs it was not compiled to understand, simply by passing the metadata for that struct. You could even write the editor in another language, if you so choose.
You can see an example of introspection for editors taken to an extreme in Unreal Engine. New structs/classes declared with UCLASS or USTRUCT prefixes automatically create editor interfaces through type introspection and UPROPERTY annotations.
Remote procedure calls
Once you have a system for easily serializing types, creating a remote-procedure call system is much easier. When an RPC is initiated, the system serializes all the arguments into a packet along with the procedure name to call. On the other side of the connection, the arguments are deserialized, the procedure is found from a function table, and then it is called.
You will likely want to use code generation for doing the argument unwrapping and forwarding to the procedure.
How it can be implemented
Some languages, like Python, provide type introspection as a built-in feature of the language. C, for example, does not provide type introspection. There are several approaches you can use to add type introspection to any language:
- Code parsing: In a stage before compiling your native language files, a separate executable parses the native language files for type definitions and generates type metadata. This generated metadata can either be stored in a custom format and loaded at runtime, or output to a new native language file (code generation) and compiled with the rest of the files. This technique is used by Unreal Engine to add various features on top of C++[0], for example
- Compiler modification: A modified version of the language's compiler could be created that understands your custom annotations and generates the type metadata. The disadvantage is complexity (maintaining a compiler) as well as portability[1]
- Separate schema: Rather than defining the types in the native language (e.g. C), define schemas in a format that is easy to parse. You then use a code generator to generate the native language type definitions as well as the metadata for the type. This is the approach used by Naughty Dog (convention link) and Google's FlatBuffers, for example. The disadvantage of this approach is the disconnect between the actual implementation - you now have a "special language" that you must write in rather than what you (and your teammates) are familiar with
The big advantage of implementing type introspection on your own is that you can customize it to your needs, whether they be specialized type declarations, different performance characteristics, et cetera.
How introspection is implemented in Cakelisp
Cakelisp has both full-power macros and arbitrary compile-time code generation and scanning, which make type introspection implementable in user-space. That is to say, it is possible to implement type introspection without having any prior support for it in Cakelisp itself.
My work-in-progress implementation of introspection is in GameLib. I added introspection to Cakelisp by creating a macro def-introspect-struct which follows regular defstruct syntax, but allows optional tags to be added to customize the metadata. Metadata is code-generated to the file by the same macro so it can be easily used at runtime. The generated C++ for my-struct, defined previously, looks like this:
1 2 3 4 5 6 7 8 9 10 | static MetadataField myStructMetadataFields[] = {
{"name", serializationTypeString, offsetof(MyStruct, name), NULL, NULL},
{"value", serializationTypeInt, offsetof(MyStruct, value), NULL, NULL},
{"decimal", serializationTypeFloat, offsetof(MyStruct, decimal), NULL, NULL},
{"truthy", serializationTypeBool, offsetof(MyStruct, truthy), NULL, NULL},
{"charry", serializationTypeChar, offsetof(MyStruct, charry), NULL, NULL}};
static MetadataStruct myStructMetadata = {
"my-struct", myStructMetadataFields,
(sizeof(myStructMetadataFields) / sizeof(myStructMetadataFields[0]))};
|
This data is never edited by hand, and automatically updates whenever the definition to my-struct updates. I will add more types, flags, and fields to the metadata format as necessary.
Conclusion
Type introspection is a great way to:
- Reduce the amount of code you need to write
- Reduce the amount of errors in your code
- Make features drastically easier to implement that would otherwise be infeasible
Next time you are copy-pasting code or dealing with boilerplate after adding a new field to a structure, consider whether type introspection could be applicable.
[0] Parsing C++ is extremely complicated, so you can decide either to support a subset of C++ when defining types or bring in a beastly dependency like libclang to help. A primary factor of my starting Cakelisp was how frustrated I was with C++'s parsing complexity. The syntax effectively encrypts the type and function signatures I need to do things like introspection.
[1] Some platforms, especially game consoles, require you to use their compiler, which they themselves modified, and likely will not release source code of.
It has been a while since I last wrote an article. Last month, I left my previous job after six years there. I moved from California to the East Coast to start at a new job, which I have been enjoying so far. The change has been especially good for showing both how much I know and how much I have to learn.
I also had several realizations about what I want to do with my career.
Goals
My current career goal is to become self-sufficient by creating products that I both believe in and own personally[1].
Cakelisp was one step towards that goal for me. I wanted to create an environment where I felt happy and empowered, two things I didn't feel with existing tools. However, recently I have realized that I had blinders on in regards to the broader, holistic view of my goal. I only focused on optimizing my development environment.
If I want to achieve my goals, I need to optimize every part of the process.
It is obvious when I say it that it was the case, but to me I had to gain that understanding. There were some resources that helped me realize this:
Escaping "just programming"
By having a clear goal to work towards, I have an easier time realizing my deficiencies. It was easy to work for years on someone else's project without learning anything about the business and production elements. It turns out those are especially important if you want to be able to make a living off of something you create.
I am making a conscious effort to practice these skills:
I'm confident in my skills in writing code that makes something that accomplishes something. Now, I need to learn what the right thing is to make and how to get it out there.
Ownership
When I was looking for a new job, it became clear how I spent the last six years contributing to someone else's value. Once I left the job, all I had was experience and some money in the bank[3]. If I was instead working on my own technology in my own business, all the time I spend on it is building value. It is similar to renting an apartment vs. buying a house—each loan payment on the house builds the equity you have, whereas renting is more akin to paying for a transient service.
When working on your own business, the trade-off is greatly increased responsibility and risk. A career is no doubt the easier option, and I will continue to have a job until I have built enough value in personal projects to take the leap.
Short-term plan
In order to actually achieve my goal, my focus is going to expand to include business-related learning. My current plan is to work on projects lasting three months each, with the goal to release something at the end of those three months to the public. The goal is to create something which could become a viable source of income. I'm trying to shift from "project" to "product" development[4].
There are several virtues to limiting project time to three months:
The time is no less than three months because I have come to realize that you cannot cheat work in nature. Very few things create large amounts of value without similarly large amounts of up-front work. The three month time period seems reasonable to create something with significant enough work invested to show some real value.
Conclusion
I continue to believe my work on Cakelisp and GameLib[5] get me closer to my goals. They do this by helping me create products faster, both thanks to their features and my increased motivation just by virtue of using them.
The best tools serve valuable purposes. My hope is I will gradually mold my tools to accomplish my broader goals, in the meantime making the tools more (provably) valuable.
[1] Besides my continual growth at my current job, which I also care about and work hard towards.
[2] In my opinion, there is a great deficit of truly good producers in the game industry. Cargo-cultism around task-tracking, Agile, stand-ups, etc. is so widely prevalent that teams have lost sight of what is actually important to project construction. I recommend everyone read McConnell's Rapid Development, which is quite pragmatic. Dig down into the why of your existing process and ask whether it's cargo-culting or actually valuable.
[3] Not to downplay the huge amount of value in both these things. I'll always be thankful for the knowledge, friendships, and opportunities gained from my work there.
[4] This doesn't mean I am selling out on my principles. I am confident a viable business can be made which still respects your freedoms (even in the Free Software sense), does not violate your privacy, has products I find interesting to develop, and has funding strategies consumers don't hate. We shall see if I can realize such a thing. The more aware of and opinionated I become on various issues, the more value I see in businesses that appeal to my principles at the expense of higher profit margins. This strategy is a viable one for those businesses, so I think I may be able to pull it off as well.
[5] To provide a brief summary, I have been working recently on foundational hash table, string dictionary, and dynamic array modules. I made a huge amount of progress on File Helper, my first three-month project. Cakelisp itself has only required minor bug fixes in the last month.
I also had several realizations about what I want to do with my career.
Goals
My current career goal is to become self-sufficient by creating products that I both believe in and own personally[1].
Cakelisp was one step towards that goal for me. I wanted to create an environment where I felt happy and empowered, two things I didn't feel with existing tools. However, recently I have realized that I had blinders on in regards to the broader, holistic view of my goal. I only focused on optimizing my development environment.
If I want to achieve my goals, I need to optimize every part of the process.
It is obvious when I say it that it was the case, but to me I had to gain that understanding. There were some resources that helped me realize this:
- The Pragmatic Programmer (Hunt & Thomas) has "Tip 2: Think! About your work", which is about always observing how you are working and considering how you might improve things
- Rapid Development (Steve McConnell) helped me realize the blind spot I had in the area of "Production", which in the game industry generally means people focused on schedules, planning, and project management. I realized the value of those skills (see footnote 2) only once I started seeing my past personal projects and their production-related failings. If you can successfully estimate, plan, develop, and deliver a product, that is one of the most generally valuable skills in the business universe (besides making the right thing)
Escaping "just programming"
By having a clear goal to work towards, I have an easier time realizing my deficiencies. It was easy to work for years on someone else's project without learning anything about the business and production elements. It turns out those are especially important if you want to be able to make a living off of something you create.
I am making a conscious effort to practice these skills:
- Finding and understanding customer needs
- Interacting with customers, for that matter
- Evaluating potential market size for a product
- Doing competitive analysis
- Preparing a marketing plan: how do I actually get people aware of the thing?
- Project management: Rapid development practice, estimation, project type (R&D vs. evolutionary prototyping vs. Strike team, etc.), and scheduling
I'm confident in my skills in writing code that makes something that accomplishes something. Now, I need to learn what the right thing is to make and how to get it out there.
Ownership
When I was looking for a new job, it became clear how I spent the last six years contributing to someone else's value. Once I left the job, all I had was experience and some money in the bank[3]. If I was instead working on my own technology in my own business, all the time I spend on it is building value. It is similar to renting an apartment vs. buying a house—each loan payment on the house builds the equity you have, whereas renting is more akin to paying for a transient service.
When working on your own business, the trade-off is greatly increased responsibility and risk. A career is no doubt the easier option, and I will continue to have a job until I have built enough value in personal projects to take the leap.
Short-term plan
In order to actually achieve my goal, my focus is going to expand to include business-related learning. My current plan is to work on projects lasting three months each, with the goal to release something at the end of those three months to the public. The goal is to create something which could become a viable source of income. I'm trying to shift from "project" to "product" development[4].
There are several virtues to limiting project time to three months:
- I continuously exercise project front- and back-end segments: idea and market evaluation on the front-end, and actually finishing something on the back-end. If every project takes a year, I gain much less experience doing these segments
- If a project ends up not finding a place in the market, I did not waste very much time on it
- A strict deadline creates a requirement for limiting scope. I naturally run wild with ideas, but putting a deadline on things makes it much easier to see what is actually important
The time is no less than three months because I have come to realize that you cannot cheat work in nature. Very few things create large amounts of value without similarly large amounts of up-front work. The three month time period seems reasonable to create something with significant enough work invested to show some real value.
Conclusion
I continue to believe my work on Cakelisp and GameLib[5] get me closer to my goals. They do this by helping me create products faster, both thanks to their features and my increased motivation just by virtue of using them.
The best tools serve valuable purposes. My hope is I will gradually mold my tools to accomplish my broader goals, in the meantime making the tools more (provably) valuable.
[1] Besides my continual growth at my current job, which I also care about and work hard towards.
[2] In my opinion, there is a great deficit of truly good producers in the game industry. Cargo-cultism around task-tracking, Agile, stand-ups, etc. is so widely prevalent that teams have lost sight of what is actually important to project construction. I recommend everyone read McConnell's Rapid Development, which is quite pragmatic. Dig down into the why of your existing process and ask whether it's cargo-culting or actually valuable.
[3] Not to downplay the huge amount of value in both these things. I'll always be thankful for the knowledge, friendships, and opportunities gained from my work there.
[4] This doesn't mean I am selling out on my principles. I am confident a viable business can be made which still respects your freedoms (even in the Free Software sense), does not violate your privacy, has products I find interesting to develop, and has funding strategies consumers don't hate. We shall see if I can realize such a thing. The more aware of and opinionated I become on various issues, the more value I see in businesses that appeal to my principles at the expense of higher profit margins. This strategy is a viable one for those businesses, so I think I may be able to pull it off as well.
[5] To provide a brief summary, I have been working recently on foundational hash table, string dictionary, and dynamic array modules. I made a huge amount of progress on File Helper, my first three-month project. Cakelisp itself has only required minor bug fixes in the last month.
I moved from the West Coast to the East Coast. It was a busy time and I still don't have a desk or chair to type at. I'm making this post to let you know why I won't be publishing Cakelisp content for a bit. I should have something for July, as I did make progress since my last article.