Notes on Software Engineering
I worked at a large software company, SAP for three years. But I didn’t get to learn much about software engineering. SAP has its own tools and process that have already dictated how things work. But over the years, I have realized how important software engineering can be. Here I outline some of the lessons I have learnt (some with a lot of pain) about this. If you are reading this and don’t know where you or ytour team stands on software engineering, take the Joel Test. This is a living list and will be kept up to date based on my experiences.
Lessons learnt
Terminal is the superior interface
Learning to use the terminal will be much more efficient in the long run. You will be able to do basic tasks as well as complex ones using just the keyboard. GUIs also take up much needed space on the screen as well as in your brain with distractions. The terminal is as distraction free as it can get when using a computer despite all the arguments against it. Learning to touch type will make this process even more efficient. I found that the Linux shell is much better to use and to script with.
Version control is a programmer’s best friend
Even if you are the only person working on a project, use version control. When working with others, a software project without version control is not professional. git is a popular one but jj also shows promise. You can also use git for any files that requires multiple revisions. This web site is also under version control. Beej’s guide is an excellent resource to understand git. Try to keep your branches and history clean. Large files are not git’s forte. Git FLS is a better choice there.
Keep it simple stupid
Write simple code. Simplicity is hard; complexity for complexity’s sake is forbidden. Keep to the basics and things will be much more easy to manage in the long run. Code is a liability; tame it and limit it. I postulate that the more you can manage to do with the fewer amount of code, the better you are as a programmer. Cognitive load is what matters. Create clean interfaces and make your program do one thing and one thing well as stated in the UNIX principles. Do the simplest thing that could possibly work.
Don’t reinvent the wheel
A lot of programming problems have already been solved. This doesn’t mean you cannot come up with an alternate solution but first examine and use the existing one. When you hit a limitation, try to improve the existing solution. Craft bespoke when it cannot be avoided. Take this with a pinch of salt though. Sometimes, creating a specific tool or piece of software is essential. As engineers we want to use our own work rather than someone else’s. This especially applies for data formats. Stay away from creating new data formats. Use existing ones like CSV, XML, HDF5 or even plain text. Watch the unreasonable effectiveness of plain text for a much better case to be made for using plain text.
To a hammer everything is a nail
Using the right tool for the right job is crucial. One could use C programming for frontend development or JavaScript for numerical computing, however the right programming language or framework usually is dictated by your software requirements. One could be really skilled in a particular language but being able to use a broad set of skills but as an engineer one has taste as well. They are different! Taste is dependent on your values. An immature engineer is inflexible in taste. Try to get an overview of the landscape and choice the best fit tool for the job.
To automate or not to automate
Automation isn’t always necessary. For activities that don’t need to be repeated, things don’t have to be done cleanly. However one needs to keep in mind that the above chart doesn’t take into account if multiple people repeat the task. Automation also brings benefit in terms of consistency (all the results looks the same) and prevents errors in processes that are repeated many time. Humans are not made to repeat the same thing over and over again, we fail at it terribly.
A bit of documentation goes a long way
Maintain at least a README file. This allows for people to understand what you are trying to do. Comments in code should describe what is being done, the code should do this. Comments are to be used to describe why something was done in a particular way. Even documentation should be limited because someone will have to maintain it. Documentation should help either the user or the developer, if not, it isn’t necessary.
Tooling defines the friction in your process
Simple, fast and open tools are usually better in most cases. They allow for quick onboarding of team members, easy deployment of work and access to a larger community of experts. Understanding your tools also goes a long way in helping improving processes. A developer should be fighting with his tools; the tool should augment the performance of the developer. Standardize tools if possible but engineers usually bring a carefully crafted and deeply personal toolset with them; account for this.