Elevator Pitch
After a major project re-write from a code base Bash/Python/C to CGO. This talk will share the experience of building a test infrastructure for complex a CGO, and will highlight the challenges we faced, and how we overcome them with Go std libraries and Go test tools.
Description
The Singularity 3.0 release introduced a significant change to the core codebase: it was rewritten from a Bash/Python/C combination into a CGO architecture. Motivated at the outset by the need to both modernize and unify the Singularity codebase, Go emerged as the obvious choice to replace the scripting languages. While advantageous in and of itself, from the perspective of software lifecycle management for example, the choice of Go offered the potential for a significant strategic upside. Briefly, it is increasingly the case that Go is the go-to language for many of the major projects in the broader container ecosystem. But this major rewrite also introduced new challenges for our QA team: how do you test a CGO project?
Facing this new challenge, the team had to build some help packages to enable features not present in the current go test std pkg. At its simplest, Singularity is a container runtime project. Runtime testing routinely requires execution of system calls (e.g., mount), plus execution of test suites that exercise systems in ways that are sometimes not allowed by regular users. Small test packages like test, for example, allow us to superset testing standards, by controlling the level of privileges on which a test function is run. Other challenges we faced include testing C library dependencies, and how to handle unit tests dependent upon installed files on different folders in the host.
This talk will focus on the challenges our team faced when building a test suite for a complex GO project, more specifically a CGO project, and how the GO standard packages can be used to supper set GO test tool for a more feature rich test framework.
Some wins we are seeing already with our test suite all based in Go * Reproducibility is important during testing, and a compiled language is much better for reproducibility. With shell tests, test results depend on many binaries on the system. * Go allows for more complex test logic to be expressed, when appropriate. For example, https://github.com/sylabs/singularity/blob/master/cmd/singularity/build_test.go is much more complete than the equivalent 2.x tests. * https://golang.org/pkg/testing/ is super powerful. Not only is it good at testing, it’s also good at benchmarking. If we want to do performance tuning at some point, benchmarking is an important aspect of that. * Having language parity between tests and the program under test is very useful. For example, say we wanted to mock out web services to increase reliability (this has been a pain point.) By using the same language, we can re-use code. Along with https://golang.org/pkg/net/http/httptest/, services can be very quickly mocked. This pattern is utilized extensively in the SCS team. * When working with formatted data (CSV, JSON), it’s difficult to understate how much easier this is with Go (https://golang.org/pkg/encoding/csv/, https://golang.org/pkg/encoding/json/) * Same goes for working with PGP keys (https://godoc.org/golang.org/x/crypto/openpgp) * Everything that can be done by a shell script can be done by Go. In fact, there are a number of enterprises using Golang for scripting (https://blog.cloudflare.com/using-go-as-a-scripting-language-in-linux/)