howdy on maladroit.dev

Australia

Tue, 05 May 2026 00:00:00 +0000

Working from Australia for the next few days. Absolutely beautiful country with some of the friendliest folks. Absolutely love the trains in Sydney and hope we get something similar in the US someday.

A few nights ago, Kayla and I ate at a very nice Japanese restaurant called JAPASU. We ended up getting the 4 course chefs tasting which included sashimi, bluefin tuna, tiger prawn, and Wagyu steak. Truly incredible experience and one of the best meals I’ve ever had. Even outside of fancy Japanese cuisine, most of the meals we have had here have been really tasty and affordable. My only complaint is that the portion sizes are way too big. We get shit in America for our portion sizes being too much. Meanwhile, over here in Australia I have been struggling to finish most meals.

More updates

Tue, 05 May 2026 00:00:00 +0000

I need to get better at blogging. I have a flow worked out for getting better at writing, but I also need to translate that into publishing. Hopefully this is a new start and the trend will continue. It really helps that I can publish from my phone now that I’m using working copy. Anyways, here’s a bunch of random thoughts I’ve written down in the last several months.

The Word of 2026 is Habits. There are a lot of things I want to fix with my life and I think the best way to go about it is forming good habits. My goal is that every month I pick up a new habit while maintaining the old habits. The habit for January is going to be unplug, no headphones in unless I’m mowing the yard or snow blowing the driveway. I want to be more attentive and less addicted to my phone and iPad. This feels like the first step in doing this.

Before I Forget

Tue, 14 Oct 2025 00:00:00 +0000

It’s been a minute since i have made one of these and honestly it’s been due to the inconvenience. I am still working on ways to make blogging frictionless, but that has been proving challenging. Especially considering I am still trying to leverage Github pages as a static site. With that all said, it’s time to shake things up a bit. One of the hardest parts about blogging is working with a blank page. The possibilities are endless and overwhelming. To counter that, I’m going to try playing around with a gimmick that will hopefully get some of the creative juices flowing. This is a little derivative of Thorsten Ball’s Register Spill format so forgive me Thorsten. Anyways, before I forget:

Initial Thoughts on JJ

Mon, 18 Aug 2025 00:00:00 +0000

I’ve been working with JJ daily for about two months now and I have some initial thoughts about it. After working with Git for over a decade, JJ was really easy to pick up.

The first thing I noticed right away was a lot of my workflow and a lot of the neovim ecosystem is very git focused. If I was going to make this work, I needed to find a suitable replacement for visual line diffs as well as a picker/finder that would let me see files that have been modified by the current revision. I wasn’t able to find existing plugins that satisfied those needs, so I wrote some minor extensions to plugins I already used. Mini.diff ware already designed with the intention to be extended to other VCS tools, so that covered diff lines. As for the picker, I took the existing git_status picker from Folke’s snacks plugin and translated it to JJ commands. It’s not perfect but it gets the job done and allows me to quickly navigate through my changes.

Bastionland

Fri, 18 Jul 2025 00:00:00 +0000

I finally got my physical copy of Mythic Bastionland in that mail and it’s fantastic. This book has been on my mind since September of last year when a friend showed it to me for the first time. I was originally pretty skeptical because I had never played an OSR game before and my only experience was with games like 5e, Blades in the Dark, and other more modern games. My biggest hang up was character progression. It felt like it must get boring after a few sessions when you don’t get cool new spells or feats that you can use in combat. I was very wrong.

Emily

Thu, 24 Apr 2025 00:00:00 +0000

A lot has changed in the last few months. My daughter Emily was born on March 22 and she’s pretty great. Paternity leave came and went in what felt like no time at all. Now I am back on the grind and trying to cut back on my phone usage entirely. While in the hospital, I watch Ludwig and Michael Reeves attempt to cross Japan with no map or smart phone in 14 days on YouTube. It left such a powerful impact on me that I have been working on trying to forgo using my phone entirely. I still use it for music and texting, but I am leaving it at the house more and more when I go out and favoring my pocket notebook instead.

Solo RPG

Sat, 01 Mar 2025 00:00:00 +0000

The first few months of 2025 have been pretty solid. I managed to learn how to play Brianstorm (kinda), I read through 100 Go Mistakes and continue to study data oriented design, and I started programming in Jai after getting access to the private beta. Sure things are a little scary with my wife having a federal position and the current administration is making her life very chaotic, but she goes on maternity leave this month and gets a well deserved break from all the churn. Not to mention we are having another kid this month which is it’s own version of scary, but I am sure we will adapt to whatever challenges are thrown our way. The year of focus is off to a ripping start with some occasional setbacks to my old ways, but I’m proud of what has been accomplished thus far.

End of 2024

Tue, 31 Dec 2024 00:00:00 +0000

The year is nearly over and that means it’s time for a new word. I can’t remember where I stole this tradition from, probably some YouTube channel or podcast I used to listen to, but each year I like to pick just one word to define the upcoming new year. I like this approach a lot more than having a resolution because it can become more fluid and change as the year goes on.

The New Sound

Fri, 20 Dec 2024 00:00:00 +0000

The last year was a pretty great year for music. From familiar favorites like King Gizzard, IDLES, and Jack White to artists brand new to me like Baker Boy, The Lemon Twigs, and Magdalena Bay. It was just a phenomenal year for albums and it was really difficult picking just one for my album of the year. I was stuck for the longest time on King Gizzard and the Lizard Wizard’s “b741” and Bring Me The Horizon’s “POST HUMAN: NeX GEn” and kept flip flopping between which I liked more. Thankfully, the burden of choosing was removed when another album came out that was so remarkable it made the decision effortless.

Focus

Sun, 08 Dec 2024 00:00:00 +0000

I saw a YouTube video earlier this week that made the claim that people today have lost the ability to focus for extended periods of time compared to people from a hundred or so years ago. I have no idea if any of that is true, but I can say from my own perspective that I have been struggling lately with staying with habits and hobbies. From weight loss to fitness to reading, it’s just been very difficult dialing in and keeping the rhythm and momentum going. I am hoping that long form journaling can at least help with this problem, that is if I can keep that habit too.

Moderation

Sun, 17 Nov 2024 00:00:00 +0000

I have been trying to be more disciplined about different aspects of my life recently. I have started intermittent fasting again to try and fit into my nicer dress clothes that I haven’t worn in years. I have also setup the app ScreenZen to moderate my YouTube usage on my phone. I used to average between 4-5 hours a day which has now been cut down to just about an hour. Not too shabby.

Study habits

Mon, 11 Nov 2024 00:00:00 +0000

I have been trying to figure out new ways to encourage studying. I have been finding it exceptionally difficult recently to read/study because life has been chaotic. The dog needs played with, Dean wants to play outside, and honestly some days I am just not motivated enough to read about some new software design technique. My latest attempt at removing friction to study is annotating PDFs in GoodNotes with the intent to refine those notes into the KB. Time will tell if that actually works or if it’s another strategy that just doesn’t stick.

New job

Sat, 02 Nov 2024 00:00:00 +0000

First week at the new job has been incredibly refreshing. Everyone is so nice and I really love the mission statement of Bloomfield Robotics. Even if it is still the honeymoon phase, it’s good to change things up and move towards discomfort rather than staying with something you know.

Despite doing a bunch of on-boarding and being mentally drained at the end of the day, I still managed to keep up the pace with Orvash. I finally fixed the sine wave bug that was plaguing me for a few weeks. The platform layer is functional to the point where the game code can begin. I’m still work-shopping some ideas on what I want Orvash to look like, and deciding what things I am going to keep from Entree.

KB

Fri, 25 Oct 2024 00:00:00 +0000

Not much gamedev progress in the last week, it’s been a bit hectic since I’m starting a new job. I have spent some time updating my knowledge-base though and thought it would be interesting to write about.

I have a notes site at kb.maladroit.dev that is GitHub pages site consisting of all my notes I have taken in the last few years. This repository was originally created for keeping my notes in Obsidian, but after a few months of that I realized I just preferred markdown instead. Some annoying hurdles to deal with were Jekyll not supporting mermaid tables, how to manage subdomains with Github, reorganizing all my notes to play nicer with jekyll, and getting a search bar working. All of which were easily fixed with a mixture of coffee, stack overflow, and swearing.

Square wave to the dome

Wed, 16 Oct 2024 00:00:00 +0000

Nothing quite as sobering as taking a 440 hz square wave right to the dome. After fiddling for just over an hour by adjusting the volume, frequency, period, and reading the new SDL3 documentation, I realized that I didn’t have my AudioStream bound to any AudioDevice.

fn SDLInitAudio() !void {
 const srcSpec = c.SDL_AudioSpec{
 .format = c.SDL_AUDIO_S16,
 .channels = 1,
 .freq = 48000,
 };
 const dstSpec = c.SDL_AudioSpec{
 .format = c.SDL_AUDIO_F32,
 .channels = 2,
 .freq = 48000,
 };

 const audioDeviceId = c.SDL_OpenAudioDevice(c.SDL_AUDIO_DEVICE_DEFAULT_PLAYBACK, &dstSpec);
 if (audioDeviceId == 0) {
 c.SDL_LogError(c.SDL_LOG_CATEGORY_APPLICATION, "%s", c.SDL_GetError());
 return error.SDLOpenAudioDevice;
 }

 audioStream = c.SDL_CreateAudioStream(&srcSpec, &dstSpec);
 if (audioStream == null) {
 c.SDL_LogError(c.SDL_LOG_CATEGORY_APPLICATION, "%s", c.SDL_GetError());
 return error.SDLCreateAudioStream;
 }

 const success = c.SDL_BindAudioStream(audioDeviceId, audioStream);
 if (!success) {
 c.SDL_LogError(c.SDL_LOG_CATEGORY_APPLICATION, "%s", c.SDL_GetError());
 return error.SDLBindAudioStream;
 }
}

Now that I have the volume lowered from 30k to 2k, I now have keyboard, gamepad, video, and sound all working for SDL3 using Zig by following alongside Casey Muratori’s Handmade Hero series.

Mon, 01 Jan 0001 00:00:00 +0000

Command Line: A modern introduction


Author	Petr Stribny
Source	Website
Published	2023

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes-documentation

Author: CNCF Source: https://kubernetes.io/docs/home/ Published: 2022

Mon, 01 Jan 0001 00:00:00 +0000

The PARA Method

Author: Tiago Forte Source: The PARA Method: The Simple System for Organizing Your Digital Life in Seconds Published: February 24, 2023

Mon, 01 Jan 0001 00:00:00 +0000

The Rust Programming Language

Author: Steve Klabnik & Carol Nichols Source: Online Published: 02.09.2023

100 Go Mistakes

Mon, 01 Jan 0001 00:00:00 +0000

100 Go Mistakes

Go is a programming language that is simple to learn but difficult to master.

Organization

Data types

Control structures

Strings

Functions and methods

Error management

Concurrency

Standard library

Testing

Optimizations

References

100 Go Mistakes

100 Go Mistakes and How to Avoid Them

Mon, 01 Jan 0001 00:00:00 +0000

100 Go Mistakes and How to Avoid Them


Author	Teiva Harsanyi
Source	Website
Published	2022

Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Algorithms

Algorithms are step by step procedures that are used for solving a computational problem. An algorithm is different from a program in that algorithms are used during the Software-design-phase. An algorithm should be independent of programming languages and are analyzed for performance. In order to design an algorithm, a designer must have domain knowledge of the problem that they are attempting to solve.

alias

Mon, 01 Jan 0001 00:00:00 +0000

alias

Creating an alias for complex commands

alias workspace=cd ~/workspace

References

Command-Line-A-modern-introduction

Any says nothing

Mon, 01 Jan 0001 00:00:00 +0000

Any says nothing

The empty interface{} or any is considered to be an over-generalization.

Mistake

Using the any or interface{} when there needs to be expression for the value in question.

Fix

Replace any with structs or generics. Only consider it when we truly do not care what the data is like with json.Marshal(value any) ([]byte, error)

References

100 Go Mistakes

Apache Iceberg

Mon, 01 Jan 0001 00:00:00 +0000

Apache Iceberg

Apache Iceberg is an open table format for huge, slow-changing, analytic data-sets. Iceberg aims to support the following:

Schema evolution with no side effects
Hidden partitioning to avoid performance penalties and user error
Partition layout evolution as data and patterns evolve over time
Time travel to reproduce historical queries
Version rollback to correct problems by resetting to a known good state
Readers isolated from concurrent writes
Writers add and remove files atomically
Planning and scanning takes O(1) and is not impacted by table size

Because Iceberg has been designed for large data-sets it can be used in production where a single table is measured in petabytes. Iceberg is also designed to solve correctness problems in eventually-consistent cloud object stores. It is cloud platform agnostic, table changes are atomic so readers never see partial changes, and concurrent writers when experiencing conflicts will retry to ensure updates are compatible.

Array

Mon, 01 Jan 0001 00:00:00 +0000

Array

An array is a collection of items that are held in order at a specific index. They are useful for storing objects of the same type in a single variable and it is very quick to access elements if you have the index. It should also be said that adding or removing objects to the middle of an array can be a slow process.

Operation	Big-O	Note
Access	O(1)
Search	O(n)
Search (sorted)	O(log(n))
Insert/Remove	O(n)	requires shifting elements by 1
Insert/Remove (at end)	O(1)	special case where no shifting is required

Common edge cases

Empty sequence
Sequence with 1 or 2 elements
Sequence with repeated elements
Duplicate values in sequence

Techniques

Assembler

Mon, 01 Jan 0001 00:00:00 +0000

Assembler

A translator of symbolic version of an instruction to binary. Assembly language is an example of a symbolic language and machine language is a binary language.

References

Computer Organization and Design

Being confused about nil and the empty slice

Mon, 01 Jan 0001 00:00:00 +0000

Being confused about nil and the empty slice

A slice is empty if it’s length is zero. A slice is nil if it is nil.

Mistake

Not being educated on the differences between the two.

Fix

A nil slice requires no allocation and because of this we should favor returning nil slices as a result

// if we don't know the length and the slice can be empty
var s []string
// another way to create a nil and empty slice that can be helpful
s := append([]int(nil), 42)
// If we know the future length
make([]string, length)
// Avoid this style unless we are initializing with elements
s := []string{}

References

100 Go Mistakes

Being confused about when to use generics

Mon, 01 Jan 0001 00:00:00 +0000

Being confused about when to use generics

Generics in Go are types that can be specified later. Before generics were added to Go, handling similar functionality across types was a chore. With generics we can leverage the following builtin type constraints:

any
comparable

Or we can define a custom constraint:

type customConstraint interface {
 ~int | ~string // this constraint only supports ints and strings
}

int is a restriction to specifically an int while ~int supports all values whose underlying type is int

Being puzzled about channel size

Mon, 01 Jan 0001 00:00:00 +0000

Being puzzled about channel size

An unbuffered channel, or a synchronous channel, has a size of zero and blocks. A buffered channel, or asynchronous channel, has some size and is unblocking until it becomes full.

For worker-pool patterns, the channel size should be the same as the number of workers. For rate-limiting, the channel size should be set in accordance to the resource we are limited by.

Outside of these use-cases, using a different channel size should be done with caution and should likely include a comment on the rationale behind that number. The smaller the value, the more CPU contention we can handle. The larger the number, the more memory we need to allocate.

Being puzzled about when to use channels or mutexes

Mon, 01 Jan 0001 00:00:00 +0000

Being puzzled about when to use channels or mutexes

In Go, channels are a communication mechanism that can be used to send and receive values. Channels can be either unbuffered, blocks until receiver is ready, or they can be buffered, blocks when the buffer is full.

For parallel goroutines, a mutex is necessary for synchronization. For coordination or communication, a channel is necessary.

References

100 Go Mistakes

Binary

Mon, 01 Jan 0001 00:00:00 +0000

Binary

Binary is a base 2 number system that is used in machine code. It can be handy at times to transform data to binary or to preform bit-wise operations.

Corner cases

Negative numbers
Overflow/underflow errors

Bit-wise operations

Check if k’th bit is set

var num uint
num & (1 << k) != 0

Set the k’th bit

var num uint
num |= (1 << k)

Clear the k’th bit

var num uint
num &= ^(1 << k)

Toggle the k’th bit

var num uint
num ^= (1 << k)

Multiply by 2^k

var num uint
num << k

Divide by 2^k

var num uint
num >> k

Check if power of 2

var num uint
(num & (num - 1)) == 0

Swap two variables

var num1 uint
var num2 uint
num1 ^= num2
num2 ^= num1
num1 ^= num2

References

Tech-Interview-Handbook

Binary search

Mon, 01 Jan 0001 00:00:00 +0000

Binary search

Given an array of integers nums which is sorted in ascending order, and an integer target, write a function to search target in nums. If target exists, then return its index. Otherwise, return -1.

You must write an algorithm with O(log n) runtime complexity.

Method

Iterate over array and change the start/end to be the mid point

Solution

go

func search(nums []int, target int) int {
	start, end := 0, len(nums) - 1
	var mid int
	for start <= end {
		mid = (end + start) / 2
		if nums[mid] < target {
			start = mid + 1
		} else if nums[mid] > target {
			end = mid - 1
		} else {
			return mid
		}
	}
	return -1
}

References

Binary search tree

Mon, 01 Jan 0001 00:00:00 +0000

Binary-search-tree

A binary-search tree (BST) is a Binary-tree that gives all the elements in order when it is traversed in order. In-order traversal being:

Left -> Root -> Right

BSTs are useful because of their time complexity.

Operation	Big-O
Access	O(log(n))
Search	O(log(n))
Insert	O(log(n))
Remove	O(log(n))

The space complexity of traversing balanced trees is O(h) where h is the height of the tree.

References

Tech-Interview-Handbook

Binary Tree

Mon, 01 Jan 0001 00:00:00 +0000

Binary tree

A binary tree is a tree where each node has a max of two children. A complete binary tree is a binary tree where every level is completely filled. A balance binary tree is a binary tree where the right and left sub-trees of every node differ by no more than 1 level.

Traversals

Given the following binary tree:

flowchart TD
	1([1])
	2([2])
	7([7])
	6([6])
	5([5])
	11([11])
	9([9])
	9a([9])
	5a([5])
	1 --> 7
	7 --> 2
	7 --> 6
	6 --> 5
	6 --> 11
	1 --> 9
	9 --> 9a
	9a --> 5a

In-order traversal: Left -> Root -> Right
- 2, 7, 5, 6, 11, 1, 9, 5, 9
Pre-order traversal: Root -> Left -> Right
- 1, 7, 2, 6, 5, 11, 9, 9, 5
Post-order traversal: Left -> Right -> Root
- 2, 5, 11, 6, 7, 5, 9, 9, 1

References

Tech-Interview-Handbook

Breadth first search

Mon, 01 Jan 0001 00:00:00 +0000

Breadth first search

Breadth-first search is a graph traversal algorithm that starts at a node and travels to all other nodes at the current depth before moving further into the graph. queue are a good data structure for tracking of the nodes that have been encountered, but not traversed.

ℹ️ Note

It is important to use a double-ended queue (not an array) due to the time complexity of dequeuing.

Implementations

go

var (
	directions = [][]int{
 {0, 1}, {0, -1}, {1, 0}, {-1, 0}
 }
)

type graph struct {
	nodes [][]int
	rows int
	cols int
}

func NewGraph(x, y int) *graph {
	g := &graph{
		nodes: make([][]int, x),
		rows: x,
		cols: y,
	}
	for i := range g.nodes {
		g.nodes[i] = make([]int, y)
	}
	return g
}

func (g *graph) bfs(startX, startY int) {
	if len(g.nodes) <= 0 {
		return
	}
	visited := make([][]bool, g.rows)
	for i := range visited {
		visited[i] = make([]bool, g.cols)
	}

	queue := [][2]int{}
	g.bfsTraverse(startX, startY, visited, queue)
}

func (g *graph) bfsTraverse(
	x, y int,
	visited [][]bool,
	queue [][2]int,
) {
	queue = append(queue, [2]int{x, y})
	for len(queue) > 0 {
		i, j := queue[0][0], queue[0][1]
		queue = queue[1:]
		if visited[i][j] == true {
			continue
		}
		visited[i][j] = true
		fmt.Printf("visited: %v, %v\n", i, j)
		for _, direction := range directions {
			nextI, nextJ := i+direction[0], j+direction[1]
			if nextI < 0 || nextI >= g.rows {
				continue
			}
			if nextJ < 0 || nextJ >= g.cols {
				continue
			}
			queue = append(queue, [2]int{nextI, nextJ})
		}
	}
}

References

Capacity planning

Mon, 01 Jan 0001 00:00:00 +0000

Capacity-planning

Capacity planning is ensuring there is sufficient capacity and redundancy to serve projected future demand. It is critical to take steps ensuring that required capacity is in place by the time it is needed.

Organic growth: Gaining new users and adoption
Inorganic growth: Adding new features or marketing campaigns

Mandatory steps

Create an accurate organic demand forecast. This should account for the lead time to acquire capacity. Factor in inorganic demand sources into the forecast.
Perform load testing of both the systems and services

References

SRE Book

cd

Mon, 01 Jan 0001 00:00:00 +0000

cd

Change directory. Works with the Directory path special characters

# navigates to the workspace directory
cd ~/workspace

# navigates to the previous directory
cd -

References

Command-Line-A-modern-introduction

Change management

Mon, 01 Jan 0001 00:00:00 +0000

Change management

~70% of outages are due to changes on live systems

Automate the following:

Progressive roll-outs
Quickly & accurately detect problems
Rolling back safely when problems occur

References

SRE-Book

Checking an error type inaccurately

Mon, 01 Jan 0001 00:00:00 +0000

Checking an error type inaccurately

Mistake

This stops working with wrapped errors

if err != nil {
 switch err := err.(type) {
 case transientError:
 http.Error(w, err.Error(), http.StatusServiceUnavailable)
 default:
 http.Error(w, err.Error(), http.StatusBadRequest)
 }
 return
}

Fix

Use the errors.As() function

if err != nil {
 if errors.As(err, &transientError{}){
 http.Error(w, err.Error(), http.StatusServiceUnavailable)
 } else {
 http.Error(w, err.Error(), http.StatusBadRequest)
 }
 return
}

References

100 Go Mistakes

Checking an error value inaccurately

Mon, 01 Jan 0001 00:00:00 +0000

Checking an error value inaccurately

Mistake

if err == sql.ErrNoRows {

}

Fix

if errors.Is(err, sql.ErrNoRows) {

}

References

100 Go Mistakes

Chris Hecker Interview Process

Mon, 01 Jan 0001 00:00:00 +0000

Chris Hecker Interview Process

The Chris Hecker interview process is an approach to interviewing software engineers that focuses on assessing the candidate based on topics they should already be very familiar with. It is designed to put the candidate into an authoritative position where they have all the answers instead of presenting them a problem they have zero familiarity with.

The interviewer’s role is to ask clarifying questions to test the candidate’s technical knowledge of projects they have worked on. This will often involve asking questions the interviewer may already know the answer to, but framing it as if they don’t. This gives the candidate the opportunity to explain a subject and demonstrate if they have a deeper technical understanding or just surface level details. It also shows off how good a candidate is at communicating ideas to someone who is unfamiliar with them.

Command line arguments

Mon, 01 Jan 0001 00:00:00 +0000

Command line arguments

Command line arguments are passed to command line applications and come in two varieties.

Positional arguments

Positional arguments can be any number of arguments that are separated by a space. If an argument contains a space in it, it must either be escaped or surrounded with quotes

cd ~/workspace

# Escaping whitespace
cd ~/workspace/ian\ weller
cd "~/workspace/ian weller"

# This is technically two positional arguments being passed
echo Hello World

Optional arguments

Arguments that are either prefixed with the short-form - or the long-form --. Optional arguments usually change the functionality of the command they are passed to. Short-form optional arguments can be separated out or combined.

Command Line Interface

Mon, 01 Jan 0001 00:00:00 +0000

Command Line Interface

A terminal used for interacting with an operating system. CLI applications can be very useful for a variety of reasons:

Interact with head-less servers
String multiple commands together using STDIN and STDOUT
Can be automated using scripting or schedulers
Easy to write

References

Command-Line-A-modern-introduction

Command substitution

Mon, 01 Jan 0001 00:00:00 +0000

Command-substitution

Command substitution replaces itself with the internal command.

python script.py --secret_key "$(cat ~/.secret_key)"

Alternatively, process substitution can be used to instead create a temporary file with the contents of the command, and then return the file path.

python script.py --file <(python generate_key.py)

References

Command-Line-A-modern-introduction

Common JSON handling mistakes

Mon, 01 Jan 0001 00:00:00 +0000

Common JSON handling mistakes

Type embedding

Mistake

This does not work because time.Time implements the json.Marshaler interface and will clobber any sort of JSON marshaling we are trying to accomplish.

type Event struct {
 ID int
 time.Time // embedded field
}

event := Event{
 ID: 123456,
 Time: time.Now(),
}

b, err := json.Marshal(event)
if err != nil {
 return err
}

Fix

type Event struct {
 ID: int
 Time: time.Time
}

Alternatively we could chose to implement the json.Marshaler interface

func (e *Event) MarshalJSON() ([]byte, error){
 return json.Marshal(
 // leveraging an anonymous struct
 struct {
 ID int
 Time time.Time
 }{
 ID: e.ID,
 Time: e.Time,
 },
 )
}

The monotonic clock

An OS handles two types of clocks:

Common SQL mistakes

Mon, 01 Jan 0001 00:00:00 +0000

Common SQL mistakes

sql.Open

Mistake

This does not always open the connection, it merely prepares it for later use.

db, err := sql.Open("mysql", dsn)
if err != nil {
 return err
}

Fix

db, err := sql.Open("mysql", dsn)
if err != nil {
 return err
}

err = db.Ping()
if err != nil {
 return err
}

Connection pooling

sql.Open doesn’t return a single connection, but a pool of connections. We can modify how this pool behaves with the following methods:

SetMaxOpenConns()
SetMaxIdleConns()
SetConnMaxIdleTime()
SetConnMaxLifetime()

Not using prepared statements

A prepared statement is a feature most SQL databases implement to execute a repeated SQL statement. This is in the interest of efficiency as well as it reduces the risk of SQL injections.

Comparing values incorrectly

Mon, 01 Jan 0001 00:00:00 +0000

Comparing values incorrectly

Comparing values in Go, using == is not always appropriate.

Mistake

== and != do not work with slices and maps.

Fix

Consider using reflect.DeepEqual(), but keep in mind this is significantly slower than ==. If performance is critical, instead compare each element of the slice/map through iteration.

For testing solutions, consider external packages like go-cmp or testify.

References

100 Go Mistakes

Compiler

Mon, 01 Jan 0001 00:00:00 +0000

Compiler

A translator of some programming language to instructions that the hardware can execute.

References

Computer-Organization-and-Design

Component Based Objects

Mon, 01 Jan 0001 00:00:00 +0000

Component Based Objects

Thinking using components instead of inheritance can prevent needlessly linking ideas that are unrelated to one another. Objects built this way can be processed by component type, rather than object instance making it easier to profile the application. Using components typically can make extending functionality easier too since they often don’t need to modify other existing components. They either extend it or provide alternatives.

Compound Objects

A compound object is a base entity instance that has a list of components. The core object merely holds the components and they communicate to each other through it. The entity usually will update by iterating over root instances rather than systems. This is a great start, but is not a fully component based approach. This way the entity is more readable, reusable, and robust allowing it to support a larger system and be reusable between projects.

Computational technique

Mon, 01 Jan 0001 00:00:00 +0000

Computational Technique

Computer Architecture

Mon, 01 Jan 0001 00:00:00 +0000

Computer Architecture

Computer Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Computer Engineering

Computer Organization and Design

Mon, 01 Jan 0001 00:00:00 +0000

Computer-Organization-and-Design


Author	David Patterson & John Hennessy
Source	Book
Published	2012

Constant time functions

Mon, 01 Jan 0001 00:00:00 +0000

Constant time functions

$$f(n)=1$$

Constant functions are the fastest function classes when describing an algorithm

References

The Algorithm Design Manual

Control

Mon, 01 Jan 0001 00:00:00 +0000

Control

The control commands the Datapath, Memory, Input and Output devices using the programming instructions given to it

References

Computer Organization and Design

Copying a sync type

Mon, 01 Jan 0001 00:00:00 +0000

Copying a sync type

All primitive types in the sync package follow one hard rule, they should never be copied.

sync.Cond
sync.Map
sync.Mutex
sync.RWMutex
sync.Once
sync.Pool
sync.WaitGroup

Mistake

This is an example of a data race from copying a sync primitive.

type Counter struct {
 mu sync.Mutex
 counters map[string]int
}

func NewCounter() Counter {
 return Counter{counters: map[string]int{}}
}

// The function receiver here is a value so it is copied instead of referenced
func (c Counter) Increment(name string){
 c.mu.Lock()
 defer c.mu.Unlock()

 c.counters[name]++
}

Fix

Alternatively, we could make Counter.mu a pointer to a sync.Mutex

CPU

Mon, 01 Jan 0001 00:00:00 +0000

CPU

The central processing unit (CPU) is the active part of a computer that is made up of the Datapath and Control

CPU execution time

CPU execution time is the time spent by the processor to complete a task

User CPU time

The time spent on the program itself

System CPU time

The time spent in the operating system on behalf of a program

References

Computer Organization and Design

Creating confusion with octal literals

Mon, 01 Jan 0001 00:00:00 +0000

Creating confusion with octal literals

Any integer literal in Go that begins with a 0 is considered to be an octal

Mistake

// This equals 108, not 110
sum := 100 + 010

Fix

Use the long form prefix 0o for describing octals

sum := 100 + 0o10

References

100 Go Mistakes

Creating data races with append

Mon, 01 Jan 0001 00:00:00 +0000

Creating data races with append

Mistake

This is a data race because the array is not full. Both goroutines will attempt to update the same index in the backing array.

s := make([]int, 0, 1)

go func() {
 s1 := append(s, 1)
}()

go func() {
 s2 := append(s, 1)
}()

Fix

Instead we can copy the slice and append to that.

s := make([]int, 0, 1)

go func() {
 sCopy := make([]int, len(s), cap(s))
 copy(sCopy, s)

 s1 := append(sCopy, 1)
}()

go func() {
 sCopy := make([]int, len(s), cap(s))
 copy(sCopy, s)

 s2 := append(sCopy, 1)
}()

References

100 Go Mistakes

Creating utility packages

Mon, 01 Jan 0001 00:00:00 +0000

Creating utility packages

Mistake

Creating any package named utils, common, or base is almost always a mistake. They become junk drawers for unrelated functions and provide no insight to what they provide.

Fix

Use specific names when naming packages and, if necessary, break up utility packages into smaller meaningful pieces.

References

100 Go Mistakes

Cubic Time Functions

Mon, 01 Jan 0001 00:00:00 +0000

Cubic time functions

f(n)=n^3

Cubic functions describe algorithms that examine all triples

References

The-Algorithm-Design-Manual

Data Oriented Design

Mon, 01 Jan 0001 00:00:00 +0000

Data Oriented Design

Data-oriented design is an approach to software development that focuses on separating the data from context and operations. It values leveraging fundamental computer science data structures and algorithms rather than modeling the problem domain in the application. The trade off is a sacrifice of human readability and initial software development speed for flexibility, maintenance, and performance.

Data oriented design is not a product or library to be added to a project, it is a technique to be learned. While data can be generic in type it is rarely generic in how it is used. Transformations can be generic, but the ordering and selection of those transformations is literally the solution to the problem.

Data Structures

Mon, 01 Jan 0001 00:00:00 +0000

Data Structures

Data-Oriented Design

Mon, 01 Jan 0001 00:00:00 +0000

Data-Oriented Design


Author	Richard Fabian
Source	Data-Oriented Design
Published	2018

Datapath

Mon, 01 Jan 0001 00:00:00 +0000

Datapath

The datapath handles arithmetic operations

References

Computer Organization and Design

Depth first search

Mon, 01 Jan 0001 00:00:00 +0000

Depth-first-search

Depth first search is a graph traversal algorithm that travels as far as possible along edges before it backtracks. This is typically accomplished using a Stack to keep track of the nodes on the current path. This could be an implicit stack through using recursion or a literal stack data structure.

Example

go

var (
	directions = [][]int{
 {0, 1}, {0, -1}, {1, 0}, {-1, 0}
 }
)

type graph struct {
	nodes [][]int
	rows int
	cols int
}

func NewGraph(x, y int) *graph {
	g := &graph{
		nodes: make([][]int, x),
		rows: x,
		cols: y,
	}
	for i := range g.nodes {
		g.nodes[i] = make([]int, y)
	}
	return g
}

func (g *graph) dfs(startX, startY int) {
	if len(g.nodes) <= 0 {
		return
	}
	visited := make([][]bool, g.rows)
	for i := range visited {
		visited[i] = make([]bool, g.cols)
	}
	g.dfsTraverse(startX, startY, visited)
}

func (g *graph) dfsTraverse(i, j int, visited [][]bool) {
	if visited[i][j] == true {
		return
	}
	visited[i][j] = true
	fmt.Printf("visited: %v, %v\n", i, j)
	for _, direction := range directions {
		nextI, nextJ := i+direction[0], j+direction[1]
		if nextI < 0 || nextI >= g.rows {
			continue
		}
		if nextJ < 0 || nextJ >= g.cols {
			continue
		}
		g.dfsTraverse(nextI, nextJ, visited)
	}
}

References

Design Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Design Patterns

Special character	Description
/	Root path
.	Symlink to current directory
..	Symlink to the parent directory
~	User directory
~iweller	The user directory for iweller

Directories	Description
/bin	binary programs
/etc	system configuration files
/home	home directory for users on the machine
/opt	third party software
/tmp	temporary space, cleared on reboot
/usr	user related programs
/var	log files

Dynamic programming

Mon, 01 Jan 0001 00:00:00 +0000

Dynamic programming

Dynamic programming is a technique for efficiently implementing a recursive algorithm by storing partial results. This is helpful because it allows for an exhaustive search without processing the same partial result over and over. It is important to know that this is a trade off of time complexity for space.

Examples

go

Before

func fib(n int) int{
	if n == 0 {
		return 0
	}
	if n == 1 {
		return 1
	}
	return fib(n-2) + fib(n-1)
}

After

const (
	miss = -1
	maxN = 92
)

var(
	cache = make([]int, maxN)
)

func fibDriver(n int) int{
	for i := range cache {
		cache[i] = miss
	}
	cache[0] = 0
	cache[1] = 1
	return fib(n)
}

func fib(n int){
	if cache[n] == miss {
		cache[n] = fib(n-2) + fib(n-1)
	}
	return cache[n]
}

References

Entity component system

Mon, 01 Jan 0001 00:00:00 +0000

Entity component system

Entity component systems simplify architecture and encourage separation of responsibilities.

Existential Processing

Mon, 01 Jan 0001 00:00:00 +0000

Existential Processing

Existential processing attempts to provide a way to remove unnecessary querying about whether or not to process data. For example, in most software it is common to check for null and the verify objects are valid before processing. This entire line of conditional checks can be removed if data could be trusted to be in a valid state before being processed.

Complexity

Cyclomatic complexity is a numeric representation of the complexity of programs, specifically concerning flow control. In short, it is:

Expecting deterministic behavior using select and channels

Mon, 01 Jan 0001 00:00:00 +0000

Expecting deterministic behavior using select and channels

Mistake

Unlike the switch case, select case is random which case will be satisfied first. If messageCh is buffered, even if we had more messages to process on the messageCh when disconnectCh receives a message it becomes random which will be triggered.

for {
 select {
 case v := <-messageCh:
 fmt.Print(v)
 case <- disconnectCh:
 return
 }
}

///////////

for i := 0; i < 10; i++ {
 messageCh <- i
}

disconnectCh <- struct{}{}

Fix

Use an unbuffered channel so that it blocks and guarantees all messages are delivered. We could also use a single channel instead of two.

Exponential time functions

Mon, 01 Jan 0001 00:00:00 +0000

Exponential time functions

f(n)=c^n

Exponential functions describe algorithms when iterating over all subsets of n items

References

The-Algorithm-Design-Manual

Factorial time functions

Mon, 01 Jan 0001 00:00:00 +0000

Factorial time functions

f(n)=n!

Factorial functions exist when generating all permutations or orderings of n items

References

The-Algorithm-Design-Manual

File

Mon, 01 Jan 0001 00:00:00 +0000

File

In Linux, everything is considered a file. General files can be anything from text, audio, pictures, etc. These can be in text or binary format. Additionally there are Directory files that serve as containers for other files. Finally device files are any IO devices connected to the computer such as disks, keyboards, etc.

References

Command-Line-A-modern-introduction

Floating points

Mon, 01 Jan 0001 00:00:00 +0000

Floating points

The concept of floating point numbers was created to solve the problem that integer values cannot represent fractional numbers. Floating point numbers are an approximation of real numbers due to the fact that their is an infinite number of fractions between two integers.

IEEE-754

The IEEE-754 standard for floats is that some bits represent a mantissa and other bits represent an exponent that is applied as a multiplier to the mantissa.

Forgetting about possible side effects with string formatting

Mon, 01 Jan 0001 00:00:00 +0000

Forgetting about possible side effects with string formatting

Mistake

func (c *Customer) String() string {
 c.mutex.RLock()
 defer c.mutex.RUnlock()
 return fmt.Sprintf("id %s, age %s", c.id, c.age)
}

func (c *Customer) UpdateAge(age int) error {
 c.mutex.Lock()
 defer c.mutex.Unlock()

 if age < 0 {
 // This creates a deadlock because it will be calling the String() method
 // through the Stringer interface
 return fmt.Errorf("age should be positive: %v", c)
 }

 c.age = age
 return nil
}

Fix

func (c *Customer) String() string {
 c.mutex.RLock()
 defer c.mutex.RUnlock()
 return fmt.Sprintf("id %s, age %s", c.id, c.age)
}

func (c *Customer) UpdateAge(age int) error {
 if age < 0 {
 return fmt.Errorf("age should be positive: %v", c)
 }

 c.mutex.Lock()
 defer c.mutex.Unlock()

 c.age = age
 return nil
}

References

100 Go Mistakes

Forgetting about sync.Cond

Mon, 01 Jan 0001 00:00:00 +0000

Forgetting about sync.Cond

Mistake

This example works but contains a CPU burner loop. It wastes a lot of cycles checking if the donation goal has been met. This could be eliminated by using a shared channel between the two goroutines, but we introduce another problem where we could surpass our goal for one goroutine while the other goroutine is using that channel.

type Donation struct {
 mu sync.RWMutex
 balance int
}
donation := &Donation{}

f := func(goal int){
 donation.mu.RLock()
 for donation.balance < goal {
 donation.mu.RUnlock()
 donation.mu.RLock()
 }
 donation.mu.RUnlock()
}
go f(10)
go f(15)

go func(){
 for {
 time.Sleep(time.Second)
 donation.mu.Lock()
 donation.balance++
 donation.mu.Unlock()
 }
}()

Fix

type Donation struct {
 cond *sync.Cond
 balance int
}
donation := &Donation{
 cond: sync.NewCond(&sync.Mutex{}),
}

f := func(goal int){
 donation.cond.L.Lock()
 for donation.balance < goal {
 // blocks until condition is met/broadcast
 // cond.Wait() does the following:
 // - Unlock the mutex
 // - Suspend the goroutine until a notification is received
 // - Lock the mutex
 donation.cond.Wait()
 }
 donation.cond.L.Unlock()
}

go f(10)
go f(15)

for {
 time.Sleep(time.Second)
 donation.cond.L.Lock()
 donation.balance++
 donation.cond.L.Unlock()
 // broadcast that a condition was met
 donation.cond.Broadcast()
}

References

100 Go Mistakes

Forgetting the return after replying to an HTTP request

Mon, 01 Jan 0001 00:00:00 +0000

Forgetting the return after replying to an HTTP request

Mistake

func handler(w http.ResponseWriter, req *http.Request) {
 err := foo(req)
 if err != nil {
 // this doesn't stop the function
 http.Error(w, "foo", http.StatusInternalServerError)
 }

 // ...
}

Fix

func handler(w http.ResponseWriter, req *http.Request) {
 err := foo(req)
 if err != nil {
 http.Error(w, "foo", http.StatusInternalServerError)
 return
 }

 // ...
}

References

100 Go Mistakes

Functional options pattern

Mon, 01 Jan 0001 00:00:00 +0000

Functional options pattern

Using closures to build up optional parameters.

Example

type (
 options struct {
 port *int
 }

 Option func(*options) error
)

func WithPort(port int) Option {
 return func(options *options) error {
 if port < 0 {
 return errors.New("port less than zero")
 }

 options.port = &port
 return nil
 }
}

func NewServer(add string, opts ...Option) (*http.Server, error){
 var options options
 for _, opt := range opts {
 err := opt(&options)
 if err != nil {
 return nil, err
 }
 }
 // remaining implementation
}

References

100-go-mistakes-and-how-to-avoid-them

git

Mon, 01 Jan 0001 00:00:00 +0000

git

worktrees

git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

References

jq manual

Go

Mon, 01 Jan 0001 00:00:00 +0000

Go

Go is a statically typed, compiled high-level programming language designed at Google. It shares similarities with C, but has memory safety, garbage collection, structural typing, and concurrency.

Graph

Mon, 01 Jan 0001 00:00:00 +0000

Graph

A graph is a collection of nodes that are connected to each other by edges. Edges can be either directed (one-way) or undirected and they can also have weights (values). A tree can be considered an undirected graph with no cycles.

Operation	Big-O
Depth-first search	O(N + E)
Breadth-first search	O(N + E)
Topological sort	O(N + E)

N being number of nodes and E being number of edges

GUI

Mon, 01 Jan 0001 00:00:00 +0000

Graphical-User-Interface

A graphical Shell

References

Command-Line-A-modern-introduction

Handling an error twice

Mon, 01 Jan 0001 00:00:00 +0000

Handling an error twice

Mistake

Logging an error in multiple places

Fix

Log or return an error, never do both.

References

100 Go Mistakes

Hash table

Mon, 01 Jan 0001 00:00:00 +0000

Hash table

A hash table is a data structure that can map keys to values using a hash function. It does this by computing an index using the hash function and stores the value in an array.

Operation	Big-O
Access	N/A
Search	O(1)
Insert	O(1)
Remove	O(1)

To be clear, the O(1) is average case as it depends on the underlying hash function, but in most instances it is safe to assume average case.

Heap

Mon, 01 Jan 0001 00:00:00 +0000

Heap

A heap is a Tree data structure that is complete, or every level is full.

Max heap: A heap where the value of the node is the greatest value in that node’s sub-tree. This must be true for all nodes in the heap
Min heap: A heap where the value of the node in the smallest value in that node’s sub-tree. This must be true for all nodes in the heap

Heaps can be valuable when it is necessary to remove nodes with either the highest or lowest priority or when inserts are mixed with removals of the root node.

Helping the Compiler

Mon, 01 Jan 0001 00:00:00 +0000

Helping the Compiler

While compilers are good at optimizing code, they are not magic wands. It is the engineer’s job to write code that the compiler can reason with and better apply it’s optimizations.

“You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.”

- Jackie Stewart

Reducing order dependence

Compilers try doing work ahead of time to increase performance. If the compiler cannot discern that the order of operations is unimportant, then it cannot do work ahead of time. When translating code to an intermediate form, some compilers will use Static Single Assignment (SSA) form. SSA assumes that a variable is never modified after initialization and instead new variables are created for any mutations. This is useful because it allows the compiler to work ahead of time, this is especially true with branches.

Hierarchical Level of Detail & Implicit State

Mon, 01 Jan 0001 00:00:00 +0000

Hierarchical Level of Detail & Implicit State

Hierarchical level of detail (HLOD) is a technique where an entity gains or loses detail depending on how relevant it currently is to the user or system. This can even be as extreme as having the entity no longer exist. Since entities can be implicit based on their components and attributes, we can optimize their performance by only loading those components that are relevant.

High level language

Mon, 01 Jan 0001 00:00:00 +0000

High level language

A programming language that can be compiled into assembly language and then assembled into machine language.

References

Computer-Organization-and-Design

history

Mon, 01 Jan 0001 00:00:00 +0000

history

History views the history of previously executed commands. Commands are kept in the $HISTFILE and are not updated until the user logs out. Updating the $HISTFILE can be forced using the command history -w.

Reuse commands

Previously executed commands can be replayed using the ! operator. This can work for both commands or specifying command parameters.

docker-compose -f ~/special-file.yml up -d

# replay the last command
!!

# replay specific command with last seen arguments
!docker-compose

Hiding from history

Adding a space to the front of any command will prevent it from being written to the $HISTFILE.

How to Become a Straigh-A Student

Mon, 01 Jan 0001 00:00:00 +0000

How to Become a Straight-A Student


Author	Cal Newport
Source	How to Become a Straight-A Student
Published	2007

Ian Weller

Mon, 01 Jan 0001 00:00:00 +0000

Bloomfield Robotics

Senior Software Engineer | October 2024 - Present

At Bloomfield Robotics, I am leading the Data platform team in designing fast and reliable APIs for our customers to consume large amounts of data. Some of my accomplishments are:

Designing and developing the Data Platform
Developing a fast Apache Parquet encoder in Go
Migrating a legacy AWS Redshift database to AWS Athena
Cultivating a pragmatic culture around development

Consider

Software Engineer | September 2023 - October 2024

At Consider, I developed and maintained web scrapers for crawling and processing data from application tracking systems. Some of my accomplishments at Consider include:

Ignoring how arguments are evaluated in range loops

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring how arguments are evaluated in range loops

When using a range loop, the provide expression is evaluated once.

Mistake

a := [3]int{0, 1, 2}
for i, v := range a {
 a[2] = 10

 if i == 2 {
 fmt.Println(v) // this will print 2, not 10
 }
}

Fix

Don’t use the range variables if you are modifying the range expression.

a := [3]int{0, 1, 2}
for i := range a {
 a[2] = 10

 if i == 2 {
 fmt.Println(a[2]) // this will print 10
 }
}

References

100 Go Mistakes

Ignoring how defer arguments and receivers are evaluated

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring how defer arguments and receivers are evaluated

Mistake

defer arguments are evaluated immediately. No matter what status is changed to it will always be an empty string for these two defer statements. The same holds true for function receivers.

func f() error {
 var status string
 defer notify(status)
 defer incrementCounter(status)
 // ...
}

Fix

Passing a pointer would allow mutations to the status variable to be reflected by defer. Using a pointer receiver would have the same effect.

Ignoring how the break statement works

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring how the break statement works

A break statement is commonly used to terminate the execution of a loop.

Mistake

	for i := 0; i < 5; i++ {
		switch i {
		default:
		case 2:
			break // this will break the switch statement not the loop
		}
	}

Fix

Using a label

loop:
	for i := 0; i < 5; i++ {
		switch i {
		default:
		case 2:
			break loop
		}
	}

References

100 Go Mistakes

Ignoring package name collisions

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring package name collisions

Package name collisions occur when a variable or function shares the same name as a package.

Mistake

Package declaration

package redis

type Client struct {
	//...
}

func NewClient() *Client {
	//...
}

func (c *Client) Get(key string) (string, error) {
	// ...
}

Client implementation

redis := redis.NewClient()
v, err := redis.Get("foo")

The redis variable now makes the redis package inaccessible in this scope

Fix

Use an import alias or change the variable name.

References

100 Go Mistakes

Ignoring the fact that elements are copied in range loops

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring the fact that elements are copied in range loops

The range keyword is a convenient way to iterate over data structures.

Mistake

range performs a copy on all the elements of the data structure.

accounts := []account{
	{balance: 100.},
	{balance: 200.},
	{balance: 300.},
}

for _, a := range accounts {
	a.balance += 1000 // This will not modify the original slice
}

Fix

accounts := []account{
	{balance: 100.},
	{balance: 200.},
	{balance: 300.},
}

// using the index for the slice
for i := range accounts {
	accounts[i].balance += 1000 // This modifies the accounts object
}

// classic for loop
for i := 0; i < len(accounts); i++ {
	accounts[i].balance += 1000 // This modifies the accounts object
}

References

100 Go Mistakes

Ignoring the impact of using pointer elements in range loops

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring the impact of using pointer elements in range loops

When not being careful, it’s possible to reference the wrong elements with range loops.

Mistake

Here we are referencing the range variable, customer, which will result in all elements of the customers slice being the same value.

func (s *Store) storeCustomers(customers []Customer) {
	for _, customer := range customers {
 // This pointer is to the range variable customer, not the customers slice
		s.m[customer.ID] = &customer
	}
}

Fix

func (s *Store) storeCustomers(customers []Customer) {
	for _, customer := range customers {
 // create a local variable first
 current := customer
		s.m[customer.ID] = &current
	}
}

References

100 Go Mistakes

Ignoring when to wrap an error

Mon, 01 Jan 0001 00:00:00 +0000

Ignoring when to wrap an error

Mistake

Returning errors without context.

Fix

Wrap errors and provide context

if err != nil {
 return fmt.Errorf("bar failed: %w", err)
}

References

100 Go Mistakes

Inaccurate string iteration

Mon, 01 Jan 0001 00:00:00 +0000

Inaccurate string iteration

Mistake

This does not print ê, but instead prints Ã and the length is printed to be 6 instead of 5. This is because the character ê requires 2 bytes to store.

s := "hêllo"

for i := range s {
 fmt.Printf("position %d: %c\n", i, s[i])
}

fmt.Printf("len=%d\n", len(s))

Fix

Use range variables

s := "hêllo"

for i, r := range s {
 fmt.Printf("position %d: %c\n", i, r)
}

Cast the string to a rune slice

s := "hêllo"
runes := []rune(s)

for i := range runes {
 fmt.Printf("position %d: %c\n", i, runes[i])
}

References

100 Go Mistakes

Index as hash key

Mon, 01 Jan 0001 00:00:00 +0000

Index as hash key

In scenarios where space needs to be preserved and a hash table cannot be used, it is possible to use an array instead.

For example, given an array of 1 to N values where N is the length of the array. The value in the array can be negated to indicate presence.

Inefficient map initialization

Mon, 01 Jan 0001 00:00:00 +0000

Inefficient map initialization

In Go a map is an unordered key value pair where all keys are distinct.

m := map[string]int {
 "one": 1,
 "two": 2,
 "three": 3,
}

Mistake

When a map grows, it doubles the number of buckets used to back it. This occurs when the average number of items in the buckets is greater than a constant value (6.5) or too many buckets have overflowed. When the map grows, all keys are dispatched again to all buckets which can become expensive with large maps.

Inefficient slice initialization

Mon, 01 Jan 0001 00:00:00 +0000

Inefficient slice initialization

When initializing a slice using make we can provide a length and an optional capacity.

Mistake

Not giving a slice enough capacity when we know it will grow in size. In Go, a slice is resized by doubling it’s capacity and allocating a new backing array.

Fix

If we know a slice will grow substantially, we should give it enough capacity to minimize the number of allocations it will require.

Input

Mon, 01 Jan 0001 00:00:00 +0000

Input

References

Computer Organization and Design

Interface

Mon, 01 Jan 0001 00:00:00 +0000

interface

Interfaces in Go provide a way to specify the behavior of an object. These abstractions are satisfied implicitly and do not need a keyword like implements to use.

type Reader interface {
 Read() []byte
}

func (io *JsonReader) Read() []byte {
 // ...
}

When to use

While not an exhaustive list, these are some of the most common cases for making use of interfaces.

Common behavior

Defining common behavior that multiple types of objects need to define. A good example of this would be sorting.

Interface on the producer side

Mon, 01 Jan 0001 00:00:00 +0000

Interface on the producer side

Deciding where to implement an interface is a pretty common mistake. An interface defined in the same package as the concrete implementation is referred to as the Producer Side. The Consumer Side is considered to be an interface defined in some external package where it is being used.

Mistake

Creating an interface on the producer side. Interfaces are satisfied implicitly in Go and because of this we should be trying to discover the abstraction, not create it. Creating an interface on the producer side means the producer is creating the abstraction for all it’s consumers.

Interface pollution

Mon, 01 Jan 0001 00:00:00 +0000

Interface pollution

Interfaces are one of the most valuable tools for structuring Go code and as a result they are often abused creating unnecessary abstractions and difficult to follow code.

Creating interfaces and abstractions early. We should not be creating an interface before their is code to implement it. Abstractions should be discovered, not created.

Common behavior

Consider the sort interface which is incredibly common

type Interface interface {
	Len() int
	Less(i, j int) bool
	Swap(i, j int)
}

Decoupling

Coupled

type CustomerService struct {
	store mysql.Store
}

func (cs CustomerService) CreateNewCustomer(id string) error {
	customer := Customer{id: id}
	return cs.store.StoreCustomer(customer)
}

Decoupled using an interface

type customerStorer interface {
	StoreCustomer(Customer) error
}

type CustomerService struct {
	storer customerStorer
}

func (cs CustomerService) CreateNewCustomer(id string) error {
	customer := Customer{id: id}
	return cs.storer.StoreCustomer(customer)
}

Restricting Behavior

Concrete implementation

type IntConfig struct {
	// ...
}

func (c *IntConfig) Get() int {
	// Retrieve configuration
}

func (c *IntConfig) Set(value int) {
	// Update configuration
}

Restricted implementation using an interface

type intConfigGetter interface {
	Get() int
}

type Foo struct {
	threshold intConfigGetter
}

func NewFoo(threshold intConfigGetter) Foo {
	return Foo{threshold: threshold}
}

func (f Foo) Bar() {
	threshold := f.threshold.Get()
	// ...
}

References

100 Go Mistakes

Interger literals

Mon, 01 Jan 0001 00:00:00 +0000

Integer literals

Integer literals are numbered values that do not have a fractional component or an exponent.

Representation

Integer literals can be represented in many different forms for convenience. The more common forms are as follows:

// These are different representations of the same literal
binary := 0b1101
decimal := 13
octal := 0o15
hexadecimal := 0x0d

Two’s complement

For signed integer literals, two’s complement is an efficient method for storing positive and negative values.

Left-most bit denotes the sign. (0 is positive, 1 is negative)
Positive values are expressed in typical binary form
Negative values are expressed in binary with the bits inverted + 1

This allows for very easy addition and subtraction operations for the processor. As an added benefit, there is also only one way to represent 0 rather than potentially having a positive and negative 0.

Interval

Mon, 01 Jan 0001 00:00:00 +0000

Interval

An interval is a subset of an array where the array consists of two element arrays. The two elements represent a start and end value.

interval := [][2]int{}

It is important when considering intervals to figure out if they should be inclusive or exclusive when it comes to overlapping and if the starting value will always be smaller than the ending value.

Corner cases

No intervals
Single interval
Two intervals
Non-overlapping intervals
Interval that completely overlaps another interval
Duplicates
Intervals that start where other intervals end

Techniques

Sort the array by starting point

This is a crucial strategy when handling interval merging

jq

Mon, 01 Jan 0001 00:00:00 +0000

jq

jq is a command line tool for processing JSON. It is a single binary, written in C, and has no runtime dependencies. This allows jq to be very fast, reliable, consistent across all machines.

References

jq manual

Kernel

Mon, 01 Jan 0001 00:00:00 +0000

Kernel

KMP

Mon, 01 Jan 0001 00:00:00 +0000

KMP Algorithm

Knuth-Morris-Pratt (KMP) is an algorithm for pattern matching in a string similar to the Sliding-window technique. The primary difference in KMP is that the algorithm does not backtrack on the text string and instead creates a lookup up index in the pattern string.

To accomplish this, a prefix look up table is generated where all the sub-strings in the pattern string that match the largest repeating prefix are given index values. This index value is used as a point of reference for where to backtrack in the pattern string when a match is not found. This allows pattern matching without the need to back track in the text string.

Linear time functions

Mon, 01 Jan 0001 00:00:00 +0000

Linear time functions

f(n)=n

Linear functions describe algorithms that need look at everything once such as a for loop.

References

The-Algorithm-Design-Manual

Linked list

Mon, 01 Jan 0001 00:00:00 +0000

Linked list

Similar to an Array, the linked list is a sequential data structure that stores elements linearly. Unlike arrays, the order is not determined by it’s position in memory, but instead it keeps an address of the next element.

This has the advantage of it being an O(1) operation to insert and delete elements in the linked list. The draw back, however, is that accessing is an O(n) operation as the linked list needs to be traveled to find the element.

Linux

Mon, 01 Jan 0001 00:00:00 +0000

Linux

Logarithmic time functions

Mon, 01 Jan 0001 00:00:00 +0000

Logarithmic time functions

f(n)=log(n)

Logarithmic functions grow very slowly as n gets large, but not as slow as Constant-functions

References

The-Algorithm-Design-Manual

Logical operators

Mon, 01 Jan 0001 00:00:00 +0000

Logical-operators

Logical operators can be used as conditional logic for connecting commands together

Operator	Outcome
&&	Will run the next command if the first one succeeded
\|\|	Will run the next command if the first one failed
;	Will run both commands regardless of the outcome

References

Command-Line-A-modern-introduction

Making wrong assumptions during map iterations

Mon, 01 Jan 0001 00:00:00 +0000

Making wrong assumptions during map iterations

There are two primary misconceptions with map iterations in Go.

Mistake

Ordering

Order is not preserved in a map in Go. Iterating over a map multiple times will likely result in a unique ordering for each iteration.

Map insert during iteration

Updating a map’s contents during iteration is allowed, but can lead to unpredictable results. When a map entry is created during iteration, it may be produced during the iteration or skipped.

Maps and memory leaks

Mon, 01 Jan 0001 00:00:00 +0000

Maps and memory leaks

When maps grow and shrink, buckets need to be added to the underlying data structure to support it.

Mistake

When shrinking a map, the elements will be garbage collected, however the buckets will still remain.

Fix

Either create a new map and copy all the elements of the old map to it. This would result in a brief window of high memory consumption considering the two maps, but the old map would be eventually collected. Alternatively, make the values pointers. Pointers take up less memory and even though the buckets will still remain, there should be significantly less of them.

Matrix

Mon, 01 Jan 0001 00:00:00 +0000

Matrix

A matrix is a 2-d array data structure that could be considered a type of graph.

Edge cases

Empty matrices
1 x 1 matrix
Matrix with only 1 row or column

Techniques

Creating an empty N x M matrix

For graph traversal it can be incredibly helpful to start with an empty matrix.

n := 9
m := 9
emptyMatrix := make([][]int, n)
for i := range emptyMatrix {
	emptyMatrix[i] = make([]int, m)
}

Copy a matrix

duplicate := make([][]int, len(matrix))
for i := range duplicate {
	duplicate[i] = make([]int, len(matrix[i]))
	copy(duplicate[i], matrix[i])
}

Transposing a matrix

Transposing a matrix is change the columns into rows and the rows into columns.

Memory

Mon, 01 Jan 0001 00:00:00 +0000

Memory

The storage area where programs exist when they are running. It is also where data is kept that is needed by running programs.

References

Computer Organization and Design

Missing code documentation

Mon, 01 Jan 0001 00:00:00 +0000

Missing code documentation

Every exported element should be documented.

Mistake

Having no documentation, or worse bad documentation

Fix

Documentation should be complete, punctuated sentences and should describe the intent, not the implementation.

Use the Deprecated: comment in documentation to indicate that this is deprecated. This is result in an LSP warning if it is being used.

Constants and variables should convey purpose and content though content can be just for internal purposes.

Misunderstanding Go contexts

Mon, 01 Jan 0001 00:00:00 +0000

Misunderstanding Go contexts

A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Deadline

A deadline is a specific point in time:

time.Duration
time.Time

If a deadline is met then any ongoing activities using that context should be stopped.

func (h publishHandler) publishPosition(position flight.Position) error {
 // context with a deadline of 4 seconds
 ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
 defer cancel() // safeguard to cancel the context goroutine

 return h.pub.Publish(ctx, position)
}

Cancellation signals

func main() {
 ctx, cancel := context.WithCancel(context.Background())
 defer cancel() // this cancel signal gets called when main exits

 go func() {
 CreateFileWatcher(ctx, "foo.txt")
 }()
 // ...
}

Context values

Key value pairs can also be passed with a context

Misusing init functions

Mon, 01 Jan 0001 00:00:00 +0000

Misusing init functions

The init() function in go is a function that is initialized first in the package. The order of evaluation is

Constants and variables
Dependency init()s in alphabetical order
Main package init()
main()

Mistake

Error management in init() functions is limited since they do not return errors. The only way to indicate that something has gone wrong in an init() is to panic. This behavior should not be left to the package to decide whether to stop an application or not.

Misusing sync.WaitGroup

Mon, 01 Jan 0001 00:00:00 +0000

Misusing sync.WaitGroup

sync.WaitGroup is a mechanism to wait for n operations to complete.

Mistake

This is a data race, there is no guarantee that another goroutine will be started before the wg.Done() method is called causing the wg.Wait() to stop blocking.

wg := sync.WaitGroup{}
var v uint64

for i := 0; i < 3; i++ {
 go func(){
 wg.Add(1)
 atomic.AddUint64(&v, 1)
 wg.Done() // this is a data race
 }()
}
wg.Wait()

Fix

We can move the wg.Add(1) outside of the goroutine or alternatively we could call wg.Add(3) outside of the for loop.

Misusing trim functions

Mon, 01 Jan 0001 00:00:00 +0000

Misusing trim functions

Mistake

TrimRight removes all trailing runes in a given set.

fmt.Println(strings.TrimRight("123oxo", "xo")) // "123"

Fix

TrimSuffix removes a trailing suffix

fmt.Println(strings.TrimRight("123oxo", "xo")) // "123o"

References

100 Go Mistakes

Mixing up concurrency and parallelism

Mon, 01 Jan 0001 00:00:00 +0000

Mixing up concurrency and parallelism

Mistake

Assuming the two are synonymous.

Fix

Parallelism is every part of a system being independent. Concurrency provides a structure to solve a problem with parts that may be parallelized.

References

100 Go Mistakes

Monitoring

Mon, 01 Jan 0001 00:00:00 +0000

Monitoring

Monitoring is the primary method to keep track of a systems health and availability. Monitoring ideally should be interpreted by software systems, not by humans.

Alerts

Alerts should only be generated when there is immediate action required.

Tickets

Tickets should only be generated when there is action required, but not urgently

Logging

Logging should be generated when there is no action required, but there needs to be a record for diagnostic purposes

Neglecting integer overflows

Mon, 01 Jan 0001 00:00:00 +0000

Neglecting integer overflows

An integer overflow occurs when an arithmetic operation creates a value outside the range that can be represented in bytes. In Go an integer overflow or underflow is silent at run-time.

Mistake

var counter int32 = math.MaxInt32
counter++ // results in -2_147_483_648

Fix

func IncInt(counter int) int {
	if counter == math.MaxInt {
		panic("int overflow")
	}
	return counter + 1
}

func AddInt(a, b int) int {
	if a > math.MaxInt-b {
		panic("int overflow")
	}
	return a + b
}

func MultiplyInt(a, b int) int {
	// If one of the operands is equal to 0, it directly returns 0.
	if a == 0 || b == 0 {
		return 0
	}

	result := a * b

	if a == 1 || b == 1 {
		// Checks if one of the return result operands is equal to 1
		return result
	}

	if a == math.MinInt || b == math.MinInt {
		// Checks if one of the operands is equal to math.MinInt
		panic("integer overflow")
	}

	if result/b != a {
		// Checks if the multiplication
		panic("integer overflow")
		// leads to an integer overflow
	}

	return result
}

References

100 Go Mistakes

Never using named result parameters

Mon, 01 Jan 0001 00:00:00 +0000

Never using named result parameters

Mistake

Name result parameters should be used when would provide better context.

type locator interface {
 getCoordinates(address string) (float32, float32, error)
}

Name result parameters should not be used when they provide no value

func ReadFull(r io.Reader, buf []byte) (n int, err error) {
 // ...
}

Fix

type locator interface {
 getCoordinates(address string) (lng, lat float32, error)
}

func ReadFull(r io.Reader, buf []byte) (int, error) {
 // ...
}

References

100 Go Mistakes

Not being aware of data alignment

Mon, 01 Jan 0001 00:00:00 +0000

Not being aware of data alignment

Data alignment is the concept of having a variable’s memory address be a multiple of its size.

byte, uint8, int8: 1 byte
uint16, int16: 2 bytes
uint32, int32, float32: 4 bytes
uint64, int64, float64: 8 bytes
complex128: 16 bytes

For all of these examples, their memory address is guaranteed to be some multiple of their size. An int32 should have an address that is a multiple of 4.

Not being aware of the possible problems with type embedding

Mon, 01 Jan 0001 00:00:00 +0000

Not being aware of the possible problems with type embedding

In Go, a struct field is embedded if it is declared without a name

type Foo struct {
 Bar
}

type Bar struct {
 Baz int
}

foo := Foo{}
foo.Baz = 42

Mistake

Type embedding can sometimes expose fields and methods that outside sources should not have access to. Consider the Lock() and Unlock() methods.

Fix

Only embed when you are confident the methods that are promoted will not cause problems. Embedding is really just about convenience and if there is any doubt it is probably best to just manually promote behaviors.

Not being careful with goroutines and loop variables

Mon, 01 Jan 0001 00:00:00 +0000

Not being careful with goroutines and loop variables

Mistake

s := []int{1, 2, 3}

for _, i := range s {
 go func() { // This is a closure and when it access `i` is not guaranteed
 fmt.Print(i)
 }()
}

Fix

s := []int{1, 2, 3}

for _, i := range s {
 // this variable is now local to each scope
 val := i
 go func() {
 fmt.Print(val)
 }()
}

s := []int{1, 2, 3}

for _, i := range s {
 // we pass the value as a parameter so it is evaluated immediately
 go func(val int) {
 fmt.Print(val)
 }(i)
}

References

100 Go Mistakes

Not categorizing tests

Mon, 01 Jan 0001 00:00:00 +0000

Not categorizing tests

Build tags

//go:build integration

package db_test

go test --tags=integration -v .

Environment variables

This provides information to the user that this test was skipped

func TestInsert(t *testing.T) {
 if os.Getenv("INTEGRATION") != true {
 t.Skip("skipping integration test")
 }
}

Short mode

func TestLongRunning(t *testing.T){
 if testing.Short() {
 t.Skip("skipping long running test")
 }
}

go test -short -v .

References

100 Go Mistakes

Not closing transient resources

Mon, 01 Jan 0001 00:00:00 +0000

Not closing transient resources

HTTP body

Mistake

type handler struct {
 client http.Client
 url string
}

func (h handler) getBody() (string, error) {
 // resp leaks here
 resp, err := h.client.Get(h.url)
 if err != nil {
 return "", err
 }

 body, err := io.ReadAll(resp.Body)
 if err != nil {
 return "", err
 }

 return string(body), nil
}

Fix

type handler struct {
 client http.Client
 url string
}

func (h handler) getBody() (string, error) {
 resp, err := h.client.Get(h.url)
 if err != nil {
 return "", err
 }

 defer func(){
 err := resp.Body.Close()
 if err != nil {
 // log
 }
 }()

 body, err := io.ReadAll(resp.Body)
 if err != nil {
 return "", err
 }

 return string(body), nil
}

sql.Rows

Mistake

// rows leaks here keeping an open connection
rows, err := db.Query("SELECT * FROM customers")
if err != nil {
 return err
}

// ...

return nil

Fix

rows, err := db.Query("SELECT * FROM customers")
if err != nil {
 return err
}

defer func(){
 err := rows.Close()
 if err != nil {
 // log
 }
}()
// ...

return nil

os.File

Mistake

This is fine for reading from a file, but for writing we want to capture any errors that may occur on the file close.

Not dealing with the time API efficiently

Mon, 01 Jan 0001 00:00:00 +0000

Not dealing with the time API efficiently

Mistake

Implementation

type Cache struct {
 mu sync.RWMutex
 events []Event
}

type Event struct {
 Timestamp time.Time
 Data string
}

func (c *Cache) TrimOlderThan(since time.Duration) {
 c.mu.Lock()
 defer c.mu.Unlock()

 // this is very difficult to test
 t := time.Now().Add(-since)
 for i := 0; i < len(c.events); i++ {
 if c.events[i].Timestamp.After(t) {
 c.events = c.events[i:]
 return
 }
 }
}

Test

func TestCache_TrimOlderThan(t *testing.T) {
 // if the machine becomes busy, we could easily miss this window
 events := []Event{
 {Timestamp: time.Now().Add(-20 * time.Millisecond)},
 {Timestamp: time.Now().Add(-10 * time.Millisecond)},
 {Timestamp: time.Now().Add(10 * time.Millisecond)},
 }

 cache := &Cache{}
 cache.Add(events)
 cache.TrimOlderThan(15 * time.Millisecond)
 got := cache.GetAll()
 want := 2

 if len(got) != want {
 t.Fatalf("want: %d, got: %d", want, len(got))
 }
}

Fix

Function type

type now func() time.Time

type Cache struct {
 mu sync.RWMutex
 events []Event
 now now
}

func NewCache() *Cache {
 return &Cache{
 events: make([]Event, 0),
 now: time.Now,
 }
}

// ...

func TestCache_TrimOlderThan(t *testing.T) {
 events := []Event{
 {Timestamp: parseTime(t, "2020-01-01T12:00:00.04Z")},
 {Timestamp: parseTime(t, "2020-01-01T12:00:00.05Z")},
 {Timestamp: parseTime(t, "2020-01-01T12:00:00.06Z")},
 }

 cache := &Cache{now: func() time.Time{
 return parseTime(t, "2020-01-01T12:00:00.06Z")
 }}
 cache.Add(events)
 cache.TrimOlderThan(15 * time.Millisecond)

 // ...

 func parseTime(t *testing.T, timestamp string) time.Time {
 // ...
 }
}

Even better is to remove the time.Duration from the parameter

Not enabling the -race flag

Mon, 01 Jan 0001 00:00:00 +0000

Not enabling the -race flag

go test -race -v .

References

100 Go Mistakes

Not exploring all the Go test features

Mon, 01 Jan 0001 00:00:00 +0000

Not exploring all the Go test features

Code coverage

go test -coverpkg=./... -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

Black box testing

package counter

package counter_test

Utility functions

func TestCustomer(t *testing.T) {
 customer := createCustomer(t, "foo")
 // ...
}

func createCustomer(t *testing.T, name string) Customer {
 // ...
 if err != nil {
 t.Fatal(err)
 }

 return customer
}

Setup and tear-down

Per function

func TestMySQLIntegration(t *testing.T) {
 // ...
 db := createConnection(t, "tcp(localhost:3306)/db")
 // ...
}

func createConnection(t *testing.T, dsn string) *sql.DB {
 db, err := sql.Open("mysql", dsn)
 if err != nil {
 t.FailNow()
 }

 t.Cleanup(
 func() {
 _ = db.Close()
 }
 )

 return db
}

Per package

func TestMain(m *testing.Main) {
 setupMySQL()
 code := m.Run()
 teardownMySQL()
 os.Exit(code)
}

References

100 Go Mistakes

Not handling an error

Mon, 01 Jan 0001 00:00:00 +0000

Not handling an error

Mistake

Completely omitting the error.

notify()

Fix

Acknowledging that an error was returned, but we are ignoring it.

_ = notify()

References

100 Go Mistakes

Not handling defer errors

Mon, 01 Jan 0001 00:00:00 +0000

Not handling defer errors

Mistake

Ignoring errors when using the defer keyword

type Closer interface {
 Close() error
}

func getBalance(db *sql.DB, clientID string) (float32, error) {
 rows, err := db.Query(query, clientID)

 if err != nil {
 return 0, err
 }

 // if this errors, it will not be reported
 defer rows.Close()

 // ...
}

Fix

Use some defer technique to ensure that the error is handled or acknowledged in some way.

type Closer interface {
 Close() error
}

func getBalance(db *sql.DB, clientID string) (float32, error) {
 rows, err := db.Query(query, clientID)

 if err != nil {
 return 0, err
 }

 defer func(){
 err := rows.Close()
 if err != nil {
 log.Printf("failed to close: %v", err)
 }
 }

 // ...
}

References

100 Go Mistakes

Not knowing how to reduce allocations

Mon, 01 Jan 0001 00:00:00 +0000

Not knowing how to reduce allocations

API changes

Mistake

Returning a slice is sharing up and causes the slice to escape the stack and get allocated on the heap.

type Reader interface {
 Read(n int) (p []byte, err error)
}

Fix

Passing the slice as a parameter is an example of sharing down and the slice is able to be allocated on the stack instead of the heap.

type Reader interface {
 Read(p []byte) (n int, err error)
}

Compiler optimizations

In the event we would want to create a map with a []byte key, we would write the following:

Not knowing which type of receiver to use

Mon, 01 Jan 0001 00:00:00 +0000

Not knowing which type of receiver to use

In Go, you can have either a pointer or a value receiver.

Mistake

This will not mutate the customer balance.

func (c customer) add(v float64) {
 c.balance += v
}

Fix

func (c *customer) add(v float64) {
 c.balance += v
}

Use pointer receivers for mutating an object. Use value receivers to keep an object immutable.

References

100 Go Mistakes

Not making slice copies correctly

Mon, 01 Jan 0001 00:00:00 +0000

Not making slice copies correctly

The builtin copy function allows copying elements from one slice to another.

Mistake

The number of elements copied to the destination slice corresponds to the minimum value between the source slice length and the destination slice length. If the destination slice is an empty slice, then 0 elements will be copied.

Fix

Use a destination slice with a length equal to or greater than the source slice

Not properly checking if a slice is empty

Mon, 01 Jan 0001 00:00:00 +0000

Not properly checking if a slice is empty

What is the clear and concise method for checking if a slice has any elements?

Mistake

This handles the nil slice, but does not work for a slice with zero elements

if slice == nil {
 return
}

Fix

if len(slice) == 0 {
 return
}

References

100 Go Mistakes

Not relying on inlining

Mon, 01 Jan 0001 00:00:00 +0000

Not relying on inlining

Inlining is replacing a function call with the body of the function, usually done automatically by the compiler. In Go, this can be observed by running go build -gcflags. This only works if the function falls within the inlining budget, otherwise the function is too complex to be inlined.

Inlining offers two benefits: removing the function call overhead and enabling more compiler optimizations. Mid-stack Inlining is inlining functions that call other functions. This is useful because it allows for fast-path inlining, distinguishing between fast and slow paths.

Not taking into account instruction-level parallelism

Mon, 01 Jan 0001 00:00:00 +0000

Not taking into account instruction-level parallelism

Instruction level parallelism (ILP) is performance optimization that allows the parallelization of a sequence of instructions. Instead of being a sum of the instructions total execution time, our total time is just the length of whichever instruction took the longest to execute.

This can be incredibly beneficial, but it can lead to hazards. If one of the instructions is dependent on another parallel instruction, like a conditional operation, this is called a control/branching hazard. CPU designers solved this problem using branch prediction. Using internal metrics, the CPU can make a best guess at what the most likely branching path will be. In the event that the CPU guesses incorrectly, the CPU will flush the current execution pipeline to ensure there are no inconsistencies leading to a performance penalty of ~10-20 cycles.

Not understanding CPU caches

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding CPU caches

CPU Architecture

Modern CPUs rely on caching to speed up memory access. In most cases there are three caching levels:

L1: ~1ns
- L1D: data
- L1I: instructions
L2: ~4ns
L3: ~10ns
Main Memory ~500-1,000ns

Additionally a CPU can be hyper-threaded, where a physical core is split into multiple logical cores (virtual cores). The L1 and L2 caches are on-die, meaning they are the same piece of silicon as the rest of the processor. The L3 cache is off-die, making it much slower than the L1 and L2.

Not understanding floating points

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding floating points

Mistake

Failing to understand the lack of precision with floating point numbers. Using conditional comparisons like == can result in unexpected behavior.

Fix

When comparing floating point numbers, check that the difference is within an acceptable range
When performing additions or subtractions, group operations with a similar order of magnitude for better accuracy.
to favor accuracy, if a sequence of operations requires addition, subtraction, multiplication, or division, perform the multiplication and division operations first.

References

100 Go Mistakes

Not understanding how the GC works

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding how the GC works

A GC keeps a tree of object references and the Go GC is based on the mark and sweep algorithm.

Mark stage: Traverse all objects on the heap and mark if they are in use
Sweep stage: Traverse the tree of reference from the root and deallocate objects that are no longer referenced

When a GC runs, it first performs a set of actions that lead to stopping the world. All available CPU time is used to perform the GC and the application is put on hold until the GC is finished. The Go GC also includes a way to free memory after consumption peak using a part of the GC called the periodic scavenger. After some time, the GC will detect that a large heap is no longer necessary and will free some of it to the OS. This can be done manually with debug.FreeOSMemory().

Not understanding race problems

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding race problems

Race problems can be among the hardest and most insidious bugs to fix.

Multiple goroutines data races

Mistake

i := 0

go func() {
 i++
}()

go func() {
 i++
}()

Fix

Using atomic operations

var i int64

go func() {
 atomic.AddInt64(&i, 1)
}()

go func() {
 atomic.AddInt64(&i, 1)
}()

Using mutexes

i := 0
mutex := sync.Mutex{}

go func() {
 mutex.Lock()
 i++
 mutex.Unlock()
}()

go func() {
 mutex.Lock()
 i++
 mutex.Unlock()
}()

Using channels

i := 0
ch := make(chan int)

go func() {
 ch <- 1
}()

go func() {
 ch <- 1
}()

i += <-ch
i += <-ch

Channel data races

Mistake

Synchronizing

i := 0
go func() {
 i++ // Racer
}()

fmt.Println(i) // Racer

Buffered channels

i := 0
ch := make(chan struct{}, 1) // buffered

go func() {
 i = 1
 <-ch // A buffered receive happens before a send
}()

ch <- struct{}{}
fmt.Println(i)

Fix

Synchronizing

i := 0
ch := make(chan struct{})

go func() {
 <-ch // this receive blocks until channel send
 fmt.Println(i)
}()

i++
// alternatively we could close this channel for the same effect
ch <- struct{}{} // channel send

Buffered channels

i := 0
ch := make(chan struct{}) // unbuffered

go func() {
 i = 1
 <-ch
}()

ch <- struct{}{}
fmt.Println(i)

References

100 Go Mistakes

Not understanding slice length and capacity

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding slice length and capacity

A slice is an array that can grow or shrink in size. In Go, a slice is backed by an array, and handles the logic of adding new elements if the backing array is full or shrinking it if it is almost empty.

Internally, a slice holds a pointer to the backing array and keeps track of a length and capacity. The length is the number of elements the slice contains, while the capacity is the number of elements in the backing array.

Not understanding stack vs heap

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding stack vs heap

Stack

The stack is the default memory, a last in; first out (LIFO) data structure that stores all the local variables for a specific goroutine. When a goroutine is started it gets 2KB of contiguous memory as it’s stack space that can grow/shrink at run-time. Moving between different functions creates a new stack frame and allocates any variables necessary in that scope. Variables outside of the stack frame still exist, but cannot be referenced in the new frame without using a pointer.

Not understanding the concept of a rune

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding the concept of a rune

In Go, a rune is a Unicode code point.

Mistake

Believing that runes are always UTF8, this is false.

Fix

Runes can be any encoding.

References

100 Go Mistakes

Not understanding the concurrency impacts of a workload type

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding the concurrency impacts of a workload type

The execution time of a workload is limited by one of the following:

CPU speed: CPU bound
I/O speed: I/O bound
Available memory: Memory bound

Mistake

How do we parallelize this task?

func read(r io.Reader) (int, error) {
	count := 0

	for {
		b := make([]byte, 1024)
		_, err := r.Read(b)

		if err != nil {
			if err == io.EOF {
				break
			}
			return 0, err
		}

		count += task(b)
	}

	return count, nil
}

Fix

Worker-pooling pattern

func read(r io.Reader) (int, error) {
	var count int64
	wg := sync.WaitGroup{}

 // What we set `n` to largely is based on whether the task is IO or CPU bound
 // For I/O bound tasks, n should be some value that the system can handle.
 // Some trial and error may be necessary to determine this value.
 // For CPU bound tasks, use GOMAXPROCS:
 // n := runtime.GOMAXPROCS(0)
	var n = 10

 // Buffered channel and wait group are the same size
	ch := make(chan []byte, n)
	wg.Add(n)

 // Create n goroutines
	for i := 0; i < n; i++ {
		go func() {
			defer wg.Done()

			for b := range ch {
				v := task(b)
				atomic.AddInt64(&count, int64(v))
			}
		}()
	}

	for {
		b := make([]byte, 1024)
		// Read from r to b
		ch <- b
	}

	close(ch)
	wg.Wait()

	return int(count), nil
}

References

100 Go Mistakes

Not understanding the impacts of running Go in Docker and Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Not understanding the impacts of running Go in Docker and Kubernetes

It is important to understand the implications of running Go in Docker/Kubernetes to prevent common situations such as CPU throttling. The environment variable GOMAXPROCS defines the limit of OS threads in charge of executing user-level code simultaneously. This defaults to the number of OS logical CPU cores. Consider the following Kubernetes spec:

spec:
 containers:
 - name: app
 image: app
 resources:
 limits:
 cpu: 4000m # 4 CPU cores

We would expect GOMAXPROCS to have a value of 4, but in this case it doesn’t. It is set to 8 which is the number of cores on the host machine. With Kubernetes Completely Fair Scheduler (CFS) our pod is given a budget of CPU time. In this case, if we were using 8 threads at max capacity when our CPU resource limit was defined as 4 cores we would have our application throttled half the time. At the time this remains an open issue in Go, but can be solved with an import in main.go:

Not using errgroup

Mon, 01 Jan 0001 00:00:00 +0000

Not using errgroup

Mistake

This code correctly handles the aggregation of results with no data races, but has no solution for error handling.

func handler (ctx context.Context, circles []Circle) ([]Result, error) {
 results := make([]Result, len(circles))
 wg := sync.WaitGroup{}
 wg.Add(len(results))

 for i, circle := range circles {
 // variable shadowing to avoid data race
 i := i
 circle := circle

 go func() {
 defer wg.Done()

 result, err := foo(ctx, circle)
 if err != nil {
 // ??????
 }
 results[i] = result
 }()
 }

 wg.Wait()
}

Fix

func handle(ctx context.Context, circles []Circle) ([]Result, error) {
 results := make([]Result, len(circles))
 g, ctx := errgroup.WithContext(ctx)

 for i, circle := range circles {
 i := i
 circle := circle

 g.Go(func() error {
 result, err := foo(ctx, circle)
 if err != nil {
 return err
 }

 results[i] = result
 return nil
 })
 }

 err := g.Wait()
 if err != nil {
 return nil, err
 }

 return results, nil
}

References

100 Go Mistakes

Not using Go diagnostic tools

Mon, 01 Jan 0001 00:00:00 +0000

Not using Go diagnostic tools

Profiling

The following profiles can be collected from the go profiler, pprof:

CPU: where the application spends its time
Goroutine: stack traces of the ongoing goroutines
Heap: heap memory allocations to monitor usage and check for leaks
Mutex: lock contentions to see mutex behavior and check if we are spending too much time in locking calls
Block: where goroutines block waiting on sync primitives

Enabling pprof

We can expose the data using the net/http/pprof package at the url http://localhost/debug/pprof. This can even be used in production. The profiles that impact performance are active for just short periods and must be enabled by default.

Not using linters

Mon, 01 Jan 0001 00:00:00 +0000

Not using linters

A linter is an automated tool for analyzing code and catching errors. Just having a linter active in your code editor will catch a variety of mistakes like variable shadowing.

Mistake

Not using a linter.

Fix

Use a linter or have code linted by CI.

References

100 Go Mistakes

Not using nil channels

Mon, 01 Jan 0001 00:00:00 +0000

Not using nil channels

Mistake

In this version, ch1 blocks ch2 until ch1 is closed. This would be a problem if ch1 stays open forever.

func merge(ch1, ch2 <-chan int) <-chan int {
 ch := make(chan int, 1)

 go func() {
 for v := range ch1 {
 ch <- v
 }

 for v := range ch2 {
 ch <- v
 }
 close(ch)
 }()

 return ch
}

Fix

Make the channel nil when it is no longer open

func merge(ch1, ch2 <- chan int) <-chan int {
 ch := make(chan int, 1)

 go func(){
 for ch1 != nil || ch2 != nil {
 select {
 case v, open := <-ch1:
 if !open {
 ch1 = nil
 break
 }
 ch <- v
 case v, open := <-ch2:
 if !open {
 ch2 = nil
 break
 }
 ch <- v
 }
 }
 close(ch)
 }()

 return ch
}

References

100 Go Mistakes

Not using notification channels

Mon, 01 Jan 0001 00:00:00 +0000

Not using notification channels

Mistake

This is 1 byte of storage

notification := make(chan bool)

Fix

This takes zero bytes of storage

notification := make(chan struct{})

References

100 Go Mistakes

Not using table driven tests

Mon, 01 Jan 0001 00:00:00 +0000

Not using table driven tests

Mistake

func removeNewLineSuffixes(s string string{
 if s == "" {
 return s
 }

 if strings.HasSuffix(s, "\r\n") {
 return removeNewLineSuffixes(s[:len(s)-2])
 }

 if strings.HasSuffix(s, "\n") {
 return removeNewLineSuffixes(s[:len(s)-1])
 }

 return s
}

func TestRemoveNewLineSuffix_Empty(t *testing.T) {
 got := removeNewLineSuffixes("")
 expected := ""
 if got != expected {
 t.Errorf("got: %s", got)
 }
}

func TestRemoveLineSuffixes_CRNL(t *testing.T) {
 got := removeNewLineSuffixes("a\r\n")
 expected := "a"
 if got != expected {
 t.Errorf("got: %s", got)
 }
}

func TestRemoveLineSuffixes_NL(t *testing.T) {
 got := removeNewLineSuffixes("a\n")
 expected := "a"
 if got != expected {
 t.Errorf("got: %s", got)
 }
}

Fix

func TestRemoveNewLineSuffix(t *testing.T) {
 tests := map[string]struct{
 input string
 want string
 }{
 "empty": {
 input: "",
 want: "",
 },
 "crnl": {
 input: "a\r\n",
 want: "a",
 },
 "nl": {
 input: "a\n",
 want: "a",
 },
 }

 for name, tt := range tests {
 t.Run(name, func(t *testing.T) {
 got := removeNewLineSuffixes(tt.input)
 if got != tt.want {
 t.Errorf("got: %s, want: %s", got, tt.want)
 }
 })
 }
}

We can also make these sub-tests run in parallel

Not using test execution modes

Mon, 01 Jan 0001 00:00:00 +0000

Not using test execution modes

Parallel

Parallel execution mode allows us to run specific tests in parallel. These tests will run alongside other parallel tests. Any sequential tests will be executed first.

func TestFoo(t *testing.T){
 t.Parallel()
}

Shuffle

Randomize the execution order.

go test -shuffle=on -v .

References

100 Go Mistakes

Not using testing utility packages

Mon, 01 Jan 0001 00:00:00 +0000

Not using testing utility packages

References

100 Go Mistakes

Not using the functional options pattern

Mon, 01 Jan 0001 00:00:00 +0000

Not using the functional options pattern

For API design, how do we handle optional configurations?

Mistake

Using struct parameters or complicated builders to handle optional configurations.

type Config struct {
	Port int
}

func NewServer(addr string, cfg Config) {
	// ...
}

// Initializes Port to 0
c1 := httplib.Config{
	Port: 0,
}

// Port is missing, so it’s initialized to 0.
c2 := httplib.Config{
}

Fix

Leverage closures and variadic arguments

type options struct {
	port *int // we make this a pointer so that it can be nil
}

// A function type that updates the options struct
type Option func(options *options) error

func WithPort(port int) Option {
	return func(options *options) error {
		if port < 0 {
			return errors.New("port should be positive")
		}

		options.port = &port

		return nil
	}
}

// Accepts many option closures
func NewServer(addr string, opts ...Option) (*http.Server, error) {
	// we create the options struct that the closures reference
	var options options

	for _, opt := range opts {
		err := opt(&options)
		if err != nil {
			return nil, err
		}
	}

	// At this stage, the options struct is built and contains the config
	// Therefore, we can implement our logic related to port configuration
	var port int
	if options.port == nil {
		port = defaultHTTPPort
	} else {
		if *options.port == 0 {
			port = randomPort()
		} else {
			port = *options.port
		}
	}
	// ...
}

References

100 Go Mistakes

notes

Mon, 01 Jan 0001 00:00:00 +0000

An ever expanding collection of notes, random facts, and strategies that I have learned over time. Notes are organized roughly following the Zettelkasten system.

How I take notes

Almost all my notes start by being written down in a notebook using the QEC Method. They are then transcribed to this knowledge-base, linked to other notes, and improved over time.

Hubs

Operating system

Mon, 01 Jan 0001 00:00:00 +0000

Operating system

An operating system is an interface between a program and the hardware. It handles input and output operations for the computer and allocates storage and memory.

References

Computer-Organization-and-Design

Optimizations

Mon, 01 Jan 0001 00:00:00 +0000

Optimizations

Optimizations cannot be done blindly, you need to know what is causing the code to run slowly. In most cases, data movement tends to be the most expensive operation while running algorithms or functions on data is less expensive. If you begin development by organizing your data into arrays, you open yourself up to many opportunities for optimization. By having such an agnostic data layout, any improvements made can be applied in many locations with little concern for incompatibility.

Output

Mon, 01 Jan 0001 00:00:00 +0000

Output

References

Computer Organization and Design

Overusing getters and setters

Mon, 01 Jan 0001 00:00:00 +0000

Overusing getters and setters

Data encapsulation refers to hiding the values and state of an object. Getters and Setters are a means to enable encapsulation, yet Go has no automatic support for getters or setters.

Mistake

Providing a getter/setter for all struct attributes when it clearly is not necessary.

Fix

Use getters/setters for the following use cases:

Encapsulating behavior for a field (validation, calculated value)
Hiding internal representation (this should rarely be considered)
Aid in debugging

References

100 Go Mistakes

Panicking

Mon, 01 Jan 0001 00:00:00 +0000

Panicking

Once a panic is triggered, it continues up the call stack until either the current goroutine has returned or panic is caught with recover.

Mistake

Panicking any time an error occurs is a mistake.

Fix

Only panic when a programming error has occurred to signal that this mistake is a bug and not a runtime error.

References

100 Go Mistakes

Pre-computation

Mon, 01 Jan 0001 00:00:00 +0000

Pre-computation

When the computational algorithm that is to be applied to a sub-array of the array is involved or complex, it can be advantageous to take advantage of pre-computation. This can take the shape of prefix/suffix operation or using hashing.

Prefix Tree

Mon, 01 Jan 0001 00:00:00 +0000

Prefix tree

An implementation of a Trie but instead of holding characters on the edges, we instead hold sub-strings of the inserted string going from left to right.

Examples

go

type TrieNode struct {
	children map[rune]*TrieNode
	isLeaf bool
}

func NewTrieNode() *TrieNode {
	tn := &TrieNode{
		children: make(map[rune]*TrieNode),
		isLeaf: false,
	}
	return tn
}

type Trie struct {
	root *TrieNode
}

func NewTrie() *Trie {
	t := &Trie{
		root: NewTrieNode(),
	}
	return t
}

func (t *Trie) Insert(key string) {
	current := t.root
	for _, index := range key {
		_, ok := current.children[index]
		if !ok {
			current.children[index] = NewTrieNode()
		}
		current = current.children[index]
	}
	current.isLeaf = true
}

func (t *Trie) Search(key string) bool {
	current := t.root
	for _, index := range key {
		_, ok := current.children[index]
		if !ok {
			return false
		}
		current = current.children[index]
	}
	return current.isLeaf
}

func (t *Trie) Prefix(key string) bool {
	current := t.root
	for _, index := range key {
		_, ok := current.children[index]
		if !ok {
			return false
		}
		current = current.children[index]
	}
	return true
}

Problem solving techniques

Mon, 01 Jan 0001 00:00:00 +0000

Problem solving techniques

When solving unfamiliar problems in software engineering, it is best to take a structured approach when breaking the problem down. In addition, make sure the problem is fully understood by gathering more information and discussing the problem with colleagues.

Visualize

Use a white board or paper to draw out diagrams or map the problem out state by state. This can prove to be critical when working with trees, graphs, matrices, and linked lists.

Programs

Mon, 01 Jan 0001 00:00:00 +0000

Programs

A program is a step by step procedure that is used for solving a computational problem. Programs are different from algorithms in that they are written during the Software implementation phase. A program must be written in a programming language and should be tested by the programmer.

Project disorganization

Mon, 01 Jan 0001 00:00:00 +0000

Project disorganization

Organizing a project in Go can be challenging. There is no enforced structure or schema which can lead to code bases that are difficult to navigate.

Mistake

Having no plan on how a project should be organized.

Fix

There a many solutions for how to layout a project that all depend on the use case.

Flat

For small projects it’s usually fine to just have a flat layout. There is no need to over-complicate things.

Propagating an inappropriate context

Mon, 01 Jan 0001 00:00:00 +0000

Propagating an inappropriate context

Mistake

This creates a race condition between the two internal requests

func handler(w http.ResponseWriter, w *http.Request) {
 response, err := doSomeTask(r.Context(), r)
 if err != nil {
 http.Error(w, err.Error(), http.StatusInternalServerError)
 return
 }

 go func() {
 err := publish(r.Context(), response) // this is a race condition
 // error handling
 }()

 writeResponse(response)
}

Fix

// We wrap the context and implement the Context interface
type detach struct {
 ctx context.Context
}

func (d detach) Deadline() (time.Time, bool) {
 return time.Time{}, false
}

func (d detach) Done() <-chan struct{} {
 return nil
}

func (d detach) Err() error {
 return nil
}

func (d detach) Value(key any) any {
 return d.ctx.Value(key)
}

func handler(w http.ResponseWriter, w *http.Request) {
 response, err := doSomeTask(r.Context(), r)
 if err != nil {
 http.Error(w, err.Error(), http.StatusInternalServerError)
 return
 }

 go func() {
 // Wrapping this detaches the cancellation signal while preserving the
 // original Values
 err := publish(detach{r.Context()}, response)
 // error handling
 }()

 writeResponse(response)
}

References

100 Go Mistakes

Providing a wrong time duration

Mon, 01 Jan 0001 00:00:00 +0000

Providing a wrong time duration

Mistake

It is not obvious when this ticker will go off. We would assume every second, but in fact it is ever microsecond.

ticker := time.NewTicker(1000)
for {
 select {
 case <-ticker.C:
 // tick
 }
}

Fix

Using the time durations provided from the time package we can be certain how this ticker works.

ticker := time.NewTicker(time.Second)
for {
 select {
 case <-ticker.C:
 // tock
 }
}

References

100 Go Mistakes

pwd

Mon, 01 Jan 0001 00:00:00 +0000

pwd

pwd is print working directory.

pwd
# /home/maladroitthief/

The environment variables $PWD and $OLDPWD can be used to reference the current and previous working directory of the current session.

References

Command-Line-A-modern-introduction

QEC Method

Mon, 01 Jan 0001 00:00:00 +0000

QEC Method of Note Taking

QEC note taking is a method described by Cal Newport in the book student which stands for question, evidence, and conclusion. It aims to make notes easy to review, identify how much evidence actually supports a claim, and make note taking fast either handwritten or typed.

Steps

Question: List questions from the text or your own questions about the material
Evidence: Read to find evidence that answers the question
Conclusion: Summarize the evidence into a statement

Example

Original Text

Catalog

Quadratic time functions

Mon, 01 Jan 0001 00:00:00 +0000

Quadratic time functions

f(n)=n^2

Quadratic functions describe algorithms that require examining all pairs of elements.

References

The-Algorithm-Design-Manual

Queue

Mon, 01 Jan 0001 00:00:00 +0000

Queue

A linear data structure that is modified by adding elements to one end and removing them from the other.

Operation	Big-O
Enqueue	O(1)
Dequeue	O(1)
Front	O(1)
Back	O(1)
isEmpty	O(1)

Examples

go

For short lived queues, the following approach can be taken. Since the memory for the slice is never returned, memory leaks may occur in long living queues.

var queue []string
// Enqueue
queue = append(queue, "Hello")
// Dequeue
queue = queue[1:]
// Front
fmt.Print(queue[0])
// Back
fmt.Print(queue[len(queue)-1])

For long lived queues, a linked list is more appropriate. The container/list package provides a doubly linked list.

Rabin-Karp

Mon, 01 Jan 0001 00:00:00 +0000

Rabin-Karp Algorithm

The Rabin-Karp algorithm is a method of finding all occurrences of a sub-string in a given string. The naive approach to this solution would be to use the Sliding-window technique. Rabin-Karp has a similar approach, but instead of comparing the window to the sub-string, it instead compares hash values using a rolling hash function.

Complexity

It has an average/best case time complexity of O(n+m) and a worse case time complexity of O(n * m). The worst case would occur if all the calculated hash values of the text were the same as the calculated hash of the pattern. It has auxiliary complexity of O(1).

RAM Model of Computing

Mon, 01 Jan 0001 00:00:00 +0000

RAM Model of Computing

The RAM Model of Computation is a model for measuring time by counting the number of steps an algorithm must take to complete. There are some assumptions made in the RAM Model such as:

All simple operations are considered to be 1 step
Loops and subroutines count for N steps with N being the number of iterations
Memory access takes 1 step

Another crucial part of the RAM Model is analyzing algorithms in multiple scenarios. The RAM Model considers:

Recursion

Mon, 01 Jan 0001 00:00:00 +0000

Recursion

Recursion is a problem solving strategy that involves breaking a problem down into smaller and smaller pieces and using those solutions collectively to find the original solution. All recursive functions have at least these two parts:

A base case where the recursion stops
Breaking the problem down and calling the function recursively

Techniques

Memoization

Some recursive functions may end up calling the same function multiple times. When possible, store the results of previously calculated functions and re-use the to improve efficiency.

Relational Data Design

Mon, 01 Jan 0001 00:00:00 +0000

Relational Data Design

Representing complex data in a computational framework

Data layout and structure is a constant trade-off between performance, readability, maintenance, future-proofing, extendability, and reuse. While more complex tables can achieve higher performance and readability, the cost becomes future proofing, maintainability, and often scalability.

Moving towards a document store type of data storage is incredibly beneficial for their simplicity and flexibility. They are more akin to file systems where documents are accessed by name and have few limitations on how they are structured. This is very good for horizontal scalability since it is easy to add more hardware when data does not need to be consistent across multiple tables/machines.

Returning a nil receiver

Mon, 01 Jan 0001 00:00:00 +0000

Returning a nil receiver

Mistake

Nil is a valid receiver in Go.

type MultiError struct {
 errs []string
}

func (m *MultiError) Add(err error) {
 m.errs = append(m.errs, err.Error())
}

// Satisfying the error interface
func (m *MultiError) Error() string {
 return strings.Join(m.errs, ";")
}

func (c Customer) Validate() error {
 var m *MultiError

 if c.Age < 0 {
 m = &MultiError{}
 m.Add(errors.New("age is negative"))
 }

 if c.Name == "" {
 if m == nil {
 m = &MultiError{}
 }

 m.Add(errors.New("name is nil"))
 }

 // Even though m is nil, it is still a valid receiver.
 // This will always return as an error
 return m
}

Fix

func (c Customer) Validate() error {
 var m *MultiError

 if c.Age < 0 {
 m = &MultiError{}
 m.Add(errors.New("age is negative"))
 }

 if c.Name == "" {
 if m == nil {
 m = &MultiError{}
 }

 m.Add(errors.New("name is nil"))
 }

 return nil
}

References

100 Go Mistakes

Returning interfaces

Mon, 01 Jan 0001 00:00:00 +0000

Returning interfaces

Returning an interface can result in dependency and import cycle hell.

Mistake

Returning an interface

Fix

Return structs and accept interfaces if at all possible. There are very few edge cases where this makes sense and it should always be heavily scrutinized.

References

100 Go Mistakes

Search Results

Mon, 01 Jan 0001 00:00:00 +0000

Searching

Mon, 01 Jan 0001 00:00:00 +0000

Searching

The greatest performance gain for searching is not having to search. If the search is unnecessary then it should be avoided. Search helpers like binary trees, hash tables, or keeping data sorted can also aid in finding rows in tables.

Indexes

The concept of an index has been around for a long time in the RDMS world. In the SQL world, learning whether a record exists would involve building a query. This query could start out simple, but overtime could become more complex depending on what data is needed. Even more complex is a query that can hook into the insertion, update, or delete operations for a table to that it could update it’s results without needing to re-run the full query.

Shell

Mon, 01 Jan 0001 00:00:00 +0000

Shell

A graphical or text interface used to interact with the operating system.

References

Command-Line-A-modern-introduction

Shell expansion

Mon, 01 Jan 0001 00:00:00 +0000

Shell-expansion

Shell expansions are shortcuts that can save a significant amount of time typing.

File globbing

Replacing a pattern in a file name or directory

Expansion	Description
~	Expansion for current home directory
*	Matches `0 TO n` characters
[ab]	Matches the characters `a OR b`
[0-9]	Matches characters in the sequence `0 TO 9`
[a-z]	Matches characters in the sequence `a TO z`
^	Negate the expansion: `^[ab]` is NOT a OR b
$	Match the end: `a$` is ends in `a`
(PATTERN1\|PATTERN2)	Match `PATTERN1 OR PATTERN2`

References

Command-Line-A-modern-introduction

Shell script

Mon, 01 Jan 0001 00:00:00 +0000

Shell script

Template

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail
if [[ "${TRACE-0}" == "1" ]]; then
	set -o xtrace
fi

if [[ "${1-}" =~ ^-*h(elp)?$ ]]; then
 echo 'Usage: ./script.sh arg-one arg-two

This is an awesome bash script to make your life better.

'
 exit
fi

cd "$(dirname "$0")"

main() {
 echo do awesome stuff
}

main "$@"

Shell variables

Mon, 01 Jan 0001 00:00:00 +0000

Shell-variables

Shell variables are variables that are only applied to the current shell instance.

var=TestVar
echo $var
echo ${var} # long format for expressions in strings

Environment variables

Environment variables are shell variables that are inherited by all system processes. A shell variable can be promoted to an environment variable by exporting it.

export var=TestVar

All current environment variables can be viewed using export

References

Command-Line-A-modern-introduction

Site Reliability Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Site Reliability Engineering

Hope is not a strategy

A site reliability engineer (SRE) is a software engineer that focuses on solving operational issues with software, creating Service-Level-Objects minimizing toil of manual tasks, reduce the cost of failure, and share ownership with developers.

Tenants

Availability
Latency
Performance
Efficiency
[[change-management]]
[[monitoring]]
[[capacity-planning]]

Principles

Embracing-risk
Eliminating-toil
Monitoring-distributed-systems
Automation
Release-engineering
Simplicity

References

SRE Book

Site Reliability Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Site Reliability Engineering


Author	Betsy Beyer, Chris Jones, Jennifer Petoff and Niall Richard Murphy
Source	Google
Published	2016

Sleeping in unit tests

Mon, 01 Jan 0001 00:00:00 +0000

Sleeping in unit tests

Mistake

Implementation

type Handler struct {
 n int
 publisher publisher
}

type publisher interface {
 Publish([]Foo)
}

func (h Handler) getBestFoo(someInputs int) Foo {
 foos := getFoos(someInputs)
 best := foos[0]

 go func() {
 if len(foos) > h.n {
 foos = foos[:h.n]
 }

 h.publisher.Publish(foos)
 }()

 return best
}

Test

type publisherMock struct {
 mu sync.RWMutex
 got []Foo
}

func (p *publisherMock) Publish(got []Foo){
 p.mu.Lock()
 defer p.mu.Unlock()

 p.got = got
}

func (p *publisherMock) Get() []Foo {
 p.mu.Lock()
 defer p.mu.Unlock()

 return p.got
}

func TestGetBestFoo(t *testing.T) {
 mock := publisherMock{}
 h := Handler{
 publisher: &mock,
 n: 2,
 }

 foo := h.getBestFoo(42)

 // this is very flaky, there is no guarantee this will be enough time
 time.Sleep(10 * time.Millisecond)

 published := mock.Get()
}

Fix

Use channels

Slice

Mon, 01 Jan 0001 00:00:00 +0000

Slice

Slices and memory leaks

Mon, 01 Jan 0001 00:00:00 +0000

Slices and memory leaks

Slicing an existing slice can lead to memory leaks under certain conditions.

Leaking capacity

Mistake

func getMessageType(msg []byte) []byte {
 return msg[:5] // This makes a slice with the same capacity as msg
}

Fix

func getMessageType(msg []byte) []byte {
 msgType := make([]byte, 5)
 copy(msgType, msg)
 return msgType
}

Slices and pointers

When working with slices, if the element is a pointer or a struct with pointer fields, it will not be collected by the garbage collector.

Sliding window

Mon, 01 Jan 0001 00:00:00 +0000

Sliding window

Sliding window is a technique that accomplishes what would normally taking multiple iterations in a single iteration. The technique can be applied when given a window of a known size, finding the result for the first window, and then shifting the window to the next position in the array or string.

Common use cases

largest sum
min/max sum
sub array/string

Examples

python

def slidingWindow(arr, windowSize):
	arrLen = len(arr)
	# outer loop needs to account for to avoid out of bounds
	for i in range(arrLen - windowSize + 1):
		# inner loop only needs to iterate over the window
		for j in range(windowSize)
			# operation inside the window
	return result

go

func slidingWindow(arr []int, windowSize int) int {
	arrLen := len(arr)
	// outer loop needs to account for to avoid out of bounds
	for i := 0; i < (arrLen - windowSize + 1); i++{
		// inner loop only needs to iterate over the window
		for j := 0; j < windowSize; j++ {
			// operation inside the window
		}
	}
	return result
}

Sorting

Mon, 01 Jan 0001 00:00:00 +0000

Sorting

Sorting is important, however it is not always necessary and can cause more performance problems than it’s worth. There are algorithms that may seem like they require sorted data, but ultimately don’t. On the other hand, some algorithms can appear to not need sorted data when in reality they do.

Be aware of how accurately the data needs to be sorted since unstable sorts are often quicker. For analog ranges, a quick sort or merge sort are slow, but will guarantee accurate sorting. For a discrete range with a large number of elements, a radix sort is the best fit. If the range of values is known at compile time, then a counting sort is a very fast two-pass sort.

Sorting and searching

Mon, 01 Jan 0001 00:00:00 +0000

Sorting and searching

Sorting is the act of rearranging a sequence in some known order such as numerical or lexicographical that is either ascending or descending. Sorting arrays are valuable as they enable binary searching which is faster than linear time.

Complexity

Sorting

Algorithm	Time	Space
Bubble sort	O(n^2)	O(1)
Insertion sort	O(n^2)	O(1)
Selection sort	O(n^2)	O(1)
Quicksort	O(n log(n))	O(log(n))
Mergesort	O(n log(n))	O(n)
Heapsort	O(n log(n))	O(1)
Counting sort	O(n + k)	O(k)
Radix sort	O(nk)	O(n + k)

Searching

Algorithm	Time
Binary search	O(log(n))

Edge cases

Empty sequence
Sequence with one element
Sequence with two elements
Sequence with duplicate elements

References

Tech-Interview-Handbook

Stack

Mon, 01 Jan 0001 00:00:00 +0000

Stack

A stack is a linear data structure that supports adding elements to one end and removing elements from that same end.

Operation	Big-O
Top/Peek	O(1)
Push	O(1)
Pop	O(1)
Search	O(n)
isEmpty	O(1)

Examples

go

For short lived stacks

var stack []string
// Push
stack = append(stack, "Hello")
// Peek
fmt.Print(stack[len(stack)-1])
// Pop
stack = stack[:len(stack)-1]

For long lived stacks

stack := list.New()
// Push
stack.PushBack("Hello")
// Peek
stack.Back()
// Pop
stack.Remove(stack.Back())

References

Tech-Interview-Handbook

Standard streams

Mon, 01 Jan 0001 00:00:00 +0000

Standard-streams

Standard in, out, and error streams created when a command is run. They are usefully for passing data between applications. Most CLI applications support using stdin as in input and will create both stdout and stderr outputs, allowing multiple commands to be chained together.

The streams behave like files and are given the following descriptors:

stdin: 0
stdout: 1
stderr: 2

These streams can be redirected using > or >> where > overwrites the file and >> appends to the end. To redirect a specific stream, prepend the operators with the stream descriptor.

Starting a goroutine without knowing when to stop it

Mon, 01 Jan 0001 00:00:00 +0000

Starting a goroutine without knowing when to stop it

Mistake

It is unclear when this goroutine will stop. The channel implementation in foo() is unknown. If this never closes then it is a leak.

ch := foo()

go func(){
 for v := range ch {
 // ...
 }
}()

Fix

We instead should be signaling resources letting them know it is time to close

func main() {
 w := newWatcher()
 defer w.close()

 // ...
}

func newWatcher() watcher {
 w := watcher{}
 go w.watch()

 return w
}

func (w watcher) close(){
 // close resource
}

References

100 Go Mistakes

String

Mon, 01 Jan 0001 00:00:00 +0000

String

Strings are character arrays and share a lot of similarities with arrays as shown in their relative time complexity.

Operation	Big-O
Access	O(1)
Search	O(n)
Insert	O(n)
Remove	O(n)

Common data-structures for looking up strings

Common string algorithms

Rabin Karp for sub-string searching
KMP for sub-string searching

Multi-string operations

Operation	Big-O	Note
Find sub-string	O(n * m)	Most naive case. KMP and Rabin Karp are more efficient
Concatenate	O(n + m)
Slice	O(m)
Split	O(n + m)	Split by using some character token
Strip	O(n)	Remove leading and trailing whitespace

Common edge cases

Empty string
String with 1 or 2 characters
String with repeated characters or sub-strings
Strings with only distinct or no repeating characters

Techniques

Counting Characters

When counting the frequency of characters or a pattern in a string, use a hash map.

Substrings and memory leaks

Mon, 01 Jan 0001 00:00:00 +0000

Substrings and memory leaks

Mistake

uuid := log[:36] // slices the string, but keeps the backing array

Fix

uuid := string([]byte(log[:36])) // copies the sub-string into a []byte
uuid := strings.Clone(log[:36]) // creates a new copy of the string

References

100 Go Mistakes

Suffix tree

Mon, 01 Jan 0001 00:00:00 +0000

Suffix tree

An implementation of a Trie but instead of holding characters on the edges, we instead hold sub-strings of the inserted string going from right to left.

Construction

A naive approach to construction a suffix tree has a complexity of O(m^2) where m is the length of the input string. This can be improved to O(m) using Ukkonen’s Suffix Tree Construction Algorithm.

Super-linear time functions

Mon, 01 Jan 0001 00:00:00 +0000

Super-linear time functions

f(n)=n*log(n)

Functions that are super-linear grow slightly faster than linear, but not by much.

References

The-Algorithm-Design-Manual

Systems software

Mon, 01 Jan 0001 00:00:00 +0000

Systems software

Systems software is software that sits between the hardware and application software of a computer. The two central types are the Operating-system and Kernel

references

computer-organization-and-design

Tech Interview Handbook

Mon, 01 Jan 0001 00:00:00 +0000

Tech Interview Handbook


Author	Yangshun Tay
Source	https://www.techinterviewhandbook.org/
Published	2018

Overview

This guide is intended to provide a quick overview of tips on how to prepare for both technical and non-technical software engineering interviews.

tee

Mon, 01 Jan 0001 00:00:00 +0000

tee

The tee command reads from stdin and writes to both stdout and one or more files.

df -h | tee disk_usage.txt

References

Command-Line-A-modern-introduction

Terminal

Mon, 01 Jan 0001 00:00:00 +0000

Terminal

A text based shell.

References

Command-Line-A-modern-introduction

The Algorithm Design Manual

Mon, 01 Jan 0001 00:00:00 +0000

The Algorithm Design Manual


Author	Steven S. Skienna
Source	The Algorithm Design Manual
Published	2020

Thinking concurrency is always faster

Mon, 01 Jan 0001 00:00:00 +0000

Thinking concurrency is always faster

Mistake

Assuming that a concurrent solution is always faster than a sequential one. If the workload we are parallelizing is too small, the overhead of creating and scheduling goroutines quickly becomes the bottle-neck.

Fix

Create a hybrid approach that only leverages goroutines when the elements being processed exceeds some threshold.

References

100 Go Mistakes

time.After and memory leaks

Mon, 01 Jan 0001 00:00:00 +0000

time.After and memory leaks

time.After(time.Duration) is the concurrent alternative for time.Sleep(time.duration).

Mistake

This would result in creating a new time.After instance that is not release until the timer expires for every iteration of this loop/every message received.

func consumer(ch <-chan Event) {
 for {
 select {
 case event := <-ch:
 // ...
 case <-time.After(time.Hour):
 // ...
 }
 }
}

Fix

This is better, but creating a new context every loop is still not great.

func consumer(ch <-chan Event) {
 for {
 ctx, cancel := context.WithTimeout(context.Background(), time.Hour)
 select {
 case event := <-ch:
 cancel()
 // ...
 case <-ctx.Done():
 // ...
 }
 }
}

The more appropriate approach would be to use a timer.

Topological sort

Mon, 01 Jan 0001 00:00:00 +0000

Topological sorting

A topological sort is a linear ordering of a directed graph. The ordering works similarly to Depth-first-search, but the nodes are considered “visited” when all of it’s dependencies are visited. This is an algorithm that is commonly used for discovering dependencies or prerequisites between nodes.

Examples

go

type graph struct {
	nodes []*node
}

type node struct {
	name string
	edges []*node
}

func NewGraph() *graph {
	return &graph{nodes: []*node{}}
}

func NewNode(name string) *node {
	return &node{name: name, edges: []*node{}}
}

func (g *graph) AddNode(n *node) {
	g.nodes = append(g.nodes, n)
}

func (n *node) AddEdge(e *node) {
	n.edges = append(n.edges, e)
}

func (g *graph) TopologicalSort() {
	if len(g.nodes) <= 0 {
		return
	}
	visited := map[*node]bool{}
	stack := []*node{}
	for _, node := range g.nodes {
		if visited[node] == false {
			stack = g.tsTraverse(node, visited, stack)
		}
	}
	for _, n := range stack {
		fmt.Println(n.name)
	}
}

func (g *graph) tsTraverse(
	node *node,
	visited map[*node]bool,
	stack []*node,
) []*node {
	visited[node] = true
	for _, edge := range node.edges {
		if visited[edge] == false {
			stack = g.tsTraverse(edge, visited, stack)
		}
	}
	return append(stack, node)
}

References

Traverse from right

Mon, 01 Jan 0001 00:00:00 +0000

Traverse-from-right

Occasionally it can be effect to traverse an array from right to left instead of left to right.

Tree

Mon, 01 Jan 0001 00:00:00 +0000

Tree

Trees are hierarchical data structures that are made up of a series of nodes. Each node can have many children but needs to have one parent. If a node does not have a parent, it is considered to be a root node.

Trees can also be considered as a undirected, connect, acyclic Graph.

Common terms

Neighbor: The parent or child nodes of the current node
Ancestor: Any node from the current node to the root node
Descendant: Any node in the current node’s subtree
Degree: Number of children at current node
Degree of a tree: Max degree of nodes in the tree
Distance: Number of edges between two nodes
Level/Depth: Number of edges between root and node
Width: Number of nodes in a level
Skewed: Nodes have very few children, like a linked list

Types of trees

Common routines

Insert
Delete
Count number of nodes
Search
Height

References

Tech-Interview-Handbook

Trie

Mon, 01 Jan 0001 00:00:00 +0000

Trie

A trie is a Tree structure where each edge represents one character and the root is null. Each path from the root represents a string, described by the characters labeling the traversed edges.

Time complexity

Operation	Big-O
Insert	O(n)
Search	O(n)

Auxiliary space

Operation	Big-O	Notes
Insert	O(m * n)	m is the Alphabet size * key length
Search	O(1)

This is a trie of “there”, “their”, “when” and “was”

Two pointers

Mon, 01 Jan 0001 00:00:00 +0000

Two pointers

The two pointers technique is a generalization of the Sliding-window technique. The primary difference is that the two pointers can cross over one another and could reference different arrays.

Common use cases

Palindromic sub-strings
Sorting colors
Merging sorted arrays

Under-optimized string concatenation

Mon, 01 Jan 0001 00:00:00 +0000

Under-optimized string concatenation

Mistake

This allocates a new string for each loop. This is due to a string being immutable in Go.

func concat(values []string) string {
 s := ""
 for _, value := range values {
 s += value
 }
 return s
}

Fix

func concat(values []string) string {
	total := 0
	// Get total number of bytes
	for i := 0; i < len(values); i++ {
		total += len(values[i])
	}

	sb := strings.Builder{}
	sb.Grow(total)
	for _, value := range values {
		_, _ = sb.WriteString(value)
	}

	return sb.String()
}

References

100 Go Mistakes

Unexpected side effects of using slice append

Mon, 01 Jan 0001 00:00:00 +0000

Unexpected side effects of using slice append

When slicing another slice, the newly created slice will have a pointer referencing the backing array of the original slice. This means that any modifications to either slice will be reflected in both.

Mistake

When appending contents to either slice that will not fit inside the backing array, a new backing array must be created. This breaks the relationship between the two slices.

Unintended side effects with named result parameters

Mon, 01 Jan 0001 00:00:00 +0000

Unintended side effects with named result parameters

Mistake

Under normal circumstances, this code would not compile because we never set nor initialize the err variable. However, since it is a named result parameter the err variable is initialized to it’s zero value and as a result err always returns nil.

func (l loc) getCoordinates(ctx context.Context, address string) (lat, lng float32, err error) {
 isValid := l.validateAddress(address)

 if !isValid {
 return 0, 0, errors.New("invalid address")
 }

 if ctx.Err() != nil {
 return 0, 0, err
 }
}

Fix

Either avoid named result parameters or shadow the err variable.

Unnecessary Nested Code

Mon, 01 Jan 0001 00:00:00 +0000

Unnecessary nested code

Multiple layers of nested code can impact readability.

Mistake

In this case, the client looks as if it is being assigned, but it is just the shadow variable that is being set.

func join(s1, s2 string, max int) (string, error) {
	if s1 == "" {
		return "", errors.New("s1 is empty")
	} else {
		if s2 == "" {
			return "", errors.New("s2 is empty")
		} else {
			// Calls a concatenate function to perform some specific concatenation but may return errors
			concat, err := concatenate(s1, s2)
			if err != nil {
				return "", err
			} else {
				if len(concat) > max {
					return concat[:max], nil
				} else {
					return concat, nil
				}
			}
		}
	}
}

Fix

Same functionality, but easier to follow

Useless string conversions

Mon, 01 Jan 0001 00:00:00 +0000

Useless string conversions

Mistake

Converting strings to []byte slice when it doesn’t make sense. While strings are human friendly, []byte slices is what most functions accept as a parameter.

Fix

Use []byte slice and the relevant transforms in the bytes package.

References

100 Go Mistakes

Using a defer inside a loop

Mon, 01 Jan 0001 00:00:00 +0000

Using a defer inside a loop

The defer statement delays execution until the surrounding function exits.

Mistake

These files could potentially be kept open forever, causing a leak.

func readFiles(ch <-chan string) error {
	for path := range ch {
		file, err := os.Open(path)

		if err != nil {
			return err
		}

		// This is deferred when readFiles exits, not the current iteration
		defer file.Close()
	}

	return nil
}

Fix

Change the function scope

func readFiles(ch <-chan string) error {
	for path := range ch {
 readFile(path)
	}

	return nil
}

func readFile(path string) error {
 file, err := os.Open(path)

 if err != nil {
 return err
 }

 defer file.Close()
}

References

100 Go Mistakes

Using a filename as a function input

Mon, 01 Jan 0001 00:00:00 +0000

Using a filename as a function input

Mistake

Testing this function would require a unique file for every use case.

func countEmptyLinesInFile(filename string) (int, error) {
 file, err := os.Open(filename)
 if err != nil {
 return 0, err
 }
 // Handle file closure
 scanner := bufio.NewScanner(file)
 for scanner.Scan() {
 // ...
 }
}

Fix

Extract the file i/o from the function logic.

func countEmptyLines(reader io.Reader) (int, error) {
 scanner := bufio.NewScanner(reader)
 for scanner.Scan() {
 // ...
 }
}

References

100 Go Mistakes

Using mutexes inaccurately with slices and maps

Mon, 01 Jan 0001 00:00:00 +0000

Using mutexes inaccurately with slices and maps

Mistake

This creates a data race. Internally a map is a runtime.hmap struct containing mostly metadata and a pointer referencing the actual data. The same holds true for slices

type Cache struct {
 mu sync.RWMutex
 balances map[string]float64
}

func (c *Cache) AddBalance(id string, balance float64) {
 c.mu.Lock()
 c.balances[id] = balance
 c.mu.Unlock()
}

func (c *Cache) AverageBalance() float64 {
 c.mu.RLock()
 // this does not copy the data, just a pointer to the data
 balances := c.balances
 c.mu.RUnlock()

 sum := 0
 for _, balance := range balances {
 sum += balance
 }

 return sum / float64(len(balances))
}

Fix

Alternatively, we could make a deep copy of the map/slice by iterating over all of it’s elements and inserting them into a new copy.

Using the default HTTP client and server

Mon, 01 Jan 0001 00:00:00 +0000

Using the default HTTP client and server

Client

Mistake

This does not specify any timeouts

client := &http.Client{}

Fix

client := &http.Client{
 Timeout: 5 * time.Second,
 Transport: &http.Transport{
 DialContext: (&net.Dialer{
 Timeout: time.Second,
 }).DialContext,
 TLSHandshakeTimeout: time.Second,
 ResponseHeaderTimeout: time.Second,
 },
}

We should also consider that the clients are pooled and re-used. Balancing these variables is critical for production facing systems.

http.Transport.IdleConnTimeout (default 90s): this is how long idle connections hang around before closed
http.Transport.MaxIdleConns (default 100): the max idle connections
http.Transport.MaxIdleConnsPerHost (default 2): max idle connections per host

Server

Mistake

server := &http.Server{}
server.Serve(listener)

Fix

server := &http.Server{
 Addr: ":8080",
 ReadHeaderTimeout: 500 * time.Millisecond,
 ReadTimeout: 500 * time.Millisecond,
 Handler: http.TimeoutHandler(handler, time.Second, "foo"),
 IdleTimeout: time.Second,
}

References

100 Go Mistakes

Variable shadowing

Mon, 01 Jan 0001 00:00:00 +0000

Variable-shadowing

Variable shadowing is re-declaring a variable in an inner block and can result in unexpected outcomes.

Mistake

In this case, the client looks as if it is being assigned, but it is just the shadow variable that is being set.

var client *http.Client
if tracing {
 // Shadowed client
	client, err := NewTracingClient()
	if err != nil {
	 return err
	}
} else {
 // Shadowed client
	client, err := NewClient()
	if err != nil {
	 return err
	}
}
// Client is still nil at this point

Fix

In this case, a temporary variable is being declared and then is assigned to the client

Writing concurrent code that leads to false sharing

Mon, 01 Jan 0001 00:00:00 +0000

Mistake

This example works from a concurrency perspective, but it is in example of false sharing and can result in poor performance. Since sumA and sumB are allocated contiguously, they more than likely will be placed on the same block. When the these goroutines end up getting scheduled to different cores, the CPU will need to copy this block of memory twice. Due to cache coherency, even though each goroutine is only modifying it’s respective sum, the CPU only tracks the cache line.

Writing inaccurate benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Writing inaccurate benchmarks

Not resetting or pausing the timer

func BenchmarkFoo(b *testing.B) {
 setup()
 b.ResetTimer()
 for i := 0; i < b.N; i++ {
 foo()
 }
}

func BenchmarkFoo(b *testing.B) {
 for i := 0; i < b.N; i++ {
 b.StopTimer()
 setup()
 b.StartTimer()
 foo()
 }
}

Micro-benchmarks

func BenchmarkAtomicStoreInt32(b *testing.B) {
 var v int32
 for i := 0; i < b.N; i++{
 atomic.StoreInt32(&v, 1)
 }
}

func BenchmarkAtomicStoreInt64(b *testing.B) {
 var v int64
 for i := 0; i < b.N; i++ {
 atomic.StoreInt64(&v, 1)
 }
}

Run the benchmark multiple times and evaluate statistics using benchstat

go test -bench=. -count=10 | stats.txt
benchstat stats.txt

Compiler optimizations

This function is likely to be in-lined, making the benchmark useless

xargs

Mon, 01 Jan 0001 00:00:00 +0000

xargs

The xargs command will run a command multiple times.

# Remove all .png files in the currect directory
find . -name "*.png" | xargs rm -rf

References

Command-Line-A-modern-introduction