For my latest hackathon at work, I decided to work on optimizing some concurrent code in one of our microservices. We had clean concurrency at one point, but then we added a branching path for a new feature and then we added another feature and then another. It quickly snowballed and got way out of hand. I restructured the code a bit and took advantage of the C# Task pattern. Azure helpfully has a Go package for C#-like async tasks and I was able to use this to simplify handling results from goroutines. Anyway, that’s not what this blog post is about. While I was trying to figure out a good way to reorganize the code, I thought it might be useful to create some sort of workflow manager that can call functions sequentially or in parallel, make decisions on which functions to call, catch errors, etc. My first inclination was to create something using a fluent interface and the builder pattern, but after reevaluating and rewriting my approach two or three times, I settled on a version consisting of chained higher-order functions. After finishing what I set out to accomplish, it turns out it wasn’t all that useful and I didn’t end up using it, but it was a fun exercise nonetheless.

To begin with, I needed a definition of a function that could perform any sort of action. Appropriately, I named it Action and it can take in any type and return any type. I also needed it to return an error so the entire workflow could terminate if an error was returned at any step along the way. Here’s the definition for the Action function:

// A simple function definition with a single input and output.
type Action func(any) (any, error)

Another fundamental building block is the ability to chain actions. I created a wrap function that accepts two actions and returns a single action. Calling the new action will execute the two provided actions in sequence. The wrap function can be called over and over to chain a bunch of actions together. If an error occurs, it will abort the sequence and return the error. Here’s what the wrap function looks like:

// Wraps provided actions so that "action" is called first and then "next" is called.
func wrap(action Action, next Action) Action {
	if action == nil && next == nil {
		return NoOp()
	}
	if action == nil {
		return next
	}
	if next == nil {
		return action
	}
	return func(in any) (any, error) {
		out, err := action(in)
		if err != nil {
			return out, err
		}
		return next(out)
	}
}

The next basic building block was to encapsulate any function with concrete types into an action. Generics are essential to making this work properly. Here’s what the Do function looks like (“do” as in “do this action”):

// Encapsulate a function with types into an action.
func Do[T1 any, T2 any](action func(T1) (T2, error)) Action {
	return func(in any) (any, error) {
		input := in.(T1)
		return action(input)
	}
}

With the Do function, it becomes easy to create actions without having to deal with type assertions. As an example, here’s a simple action that accepts an integer, adds +1 to the input, and returns the sum:

add := Do(func(in int) (int, error) {
	return in + 1, nil
})

With these basic building blocks in place, it then became trivial to add other functions that operate on actions. Here are some of those functions:

  • Do: Perform an action. Takes a function and wraps it in the Action type.
  • Sequential: Perform some actions in sequence.
  • Parallel: Perform some actions in parallel.
  • If: Conditionally perform one action or another.
  • NoOp: Does nothing. Useful as a dead end.
  • Catch: Handle an error instead of terminating the workflow.
  • Finally: Call a follow-up function after an action completes, regardless of whether or not an error occurred.
  • Retry: Retry an action if an error occurs.

These functions can be chained together to construct a full workflow, like the following example:

action := Sequential(
    Do(add1),
    Parallel(sum,
        Do(add1),
        Do(add2),
        Sequential(
            Do(add1),
            Do(add2),
            If(isOdd,
                Do(add2),
                Do(add3),
            ),
        )),
    If(isOdd,
        NoOp(),
        Do(add2),
    ),
)

Calling the resulting function will execute the entire workflow:

result, err := action(1)

The full code for this experiment can be found here: https://github.com/eleniums/go-workflow

In the end, it wasn’t useful to extract our code into an in-memory workflow for various reasons I won’t get into here, so I moved on. However, the code was fun to play around with and that’s one good reason to participate in hackathons. I regret nothing.

AI cover artwork generated by NightCafe Studio