Robust Go Web App Testing Strategies: From Unit to Dockerized Integration

Writing robust and maintainable web applications in Go demands a solid testing strategy. In today's fast-paced development environment, where applications often integrate with numerous external services and databases, a well-defined testing pyramid is not just a best practice—it's a necessity. It ensures that code changes don't introduce regressions, that new features work as expected, and that the application behaves reliably under various conditions. This article will guide you through a comprehensive testing approach for Go web applications, starting from the granular level of unit tests and progressively moving towards full-stack integration tests utilizing the power of Docker.

Understanding the Testing Landscape

Before diving into implementation, let's briefly define the core testing terminology essential for building resilient Go applications:

• Unit Tests: These are the smallest and fastest tests, focusing on isolated pieces of code, typically individual functions or methods. Their goal is to verify that each unit of code performs its intended task correctly, independent of external dependencies.
• Integration Tests: These tests aim to verify the interactions between different components or modules of an application. This could involve testing the communication between your service and a database, an external API, or other internal microservices. Integration tests are typically slower than unit tests as they involve more setup and external resources.
• End-to-End (E2E) Tests: These tests simulate a real user's journey through the application, from start to finish. They validate the entire system, including the UI, backend logic, and all integrated services, ensuring the application meets business requirements. While crucial, they are the slowest and most complex to maintain.
• Mocks: In testing, a mock object is a simulated object that mimics the behavior of a real dependency. Mocks are commonly used in unit testing to isolate the code under test from its external dependencies, allowing you to control the dependency's behavior and avoid slow or unpredictable external calls.
• Test Doubles: A general term for any kind of object used in testing to stand in for a real object. Mocks are a specific type of test double, along with stubs, fakes, and spies.
• Test Fixtures: A fixed state or environment used as a baseline for running tests. This can include setting up databases, configuring external services, or populating initial data.

Unit Testing Go Web Applications

Go provides powerful built-in tools for writing unit tests. The testing package is at the heart of it, offering a simple yet effective framework.

A typical Go web application often involves handlers, services, and repositories. Let's consider a simple HTTP handler that interacts with a user service.

// user_service.go
package main

import "errors"

type User struct {
	ID   string
	Name string
}

// UserService defines the interface for user-related operations
type UserService interface {
	GetUserByID(id string) (*User, error)
	CreateUser(name string) (*User, error)
}

// MockUserService for testing
type MockUserService struct {
	GetUserByIDFunc func(id string) (*User, error)
	CreateUserFunc  func(name string) (*User, error)
}

func (m *MockUserService) GetUserByID(id string) (*User, error) {
	if m.GetUserByIDFunc != nil {
		return m.GetUserByIDFunc(id)
	}
	return nil, errors.New("not implemented")
}

func (m *MockUserService) CreateUser(name string) (*User, error) {
	if m.CreateUserFunc != nil {
		return m.CreateUserFunc(name)
	}
	return nil, errors.New("not implemented")
}

Now, let's write an HTTP handler that uses this service.

// handlers.go
package main

import (
	"encoding/json"
	"fmt"
	"net/http"
)

type UserHandler struct {
	Service UserService
}

func (h *UserHandler) GetUser(w http.ResponseWriter, r *http.Request) {
	userID := r.URL.Query().Get("id")
	if userID == "" {
		http.Error(w, "User ID is required", http.StatusBadRequest)
		return
	}

	user, err := h.Service.GetUserByID(userID)
	if err != nil {
		http.Error(w, "User not found", http.StatusNotFound)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	json.NewEncoder(w).Encode(user)
}

func (h *UserHandler) CreateUser(w http.ResponseWriter, r *http.Request) {
	var user User
	err := json.NewDecoder(r.Body).Decode(&user)
	if err != nil {
		http.Error(w, "Invalid request body", http.StatusBadRequest)
		return
	}

	createdUser, err := h.Service.CreateUser(user.Name)
	if err != nil {
		http.Error(w, "Failed to create user", http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusCreated)
	json.NewEncoder(w).Encode(createdUser)
}

Unit Testing UserHandler

To test UserHandler, we need to mock the UserService dependency. This ensures that our handler test doesn't depend on a real database or external service.

// handlers_test.go
package main

import (
	"bytes"
	"encoding/json"
	"errors"
	"net/http"
	"net/http/httptest"
	"testing"
)

func TestUserHandler_GetUser(t *testing.T) {
	tests := []struct {
		name         string
		userID       string
		mockService  *MockUserService
		expectedCode int
		expectedBody string
	}{
		{
			name:   "Successful Get",
			userID: "123",
			mockService: &MockUserService{
				GetUserByIDFunc: func(id string) (*User, error) {
					if id == "123" {
						return &User{ID: "123", Name: "Alice"}, nil
					}
					return nil, errors.New("user not found")
				},
			},
			expectedCode: http.StatusOK,
			expectedBody: `{"ID":"123","Name":"Alice"}`,
		},
		{
			name:   "User Not Found",
			userID: "456",
			mockService: &MockUserService{
				GetUserByIDFunc: func(id string) (*User, error) {
					return nil, errors.New("user not found")
				},
			},
			expectedCode: http.StatusNotFound,
			expectedBody: `User not found` + "\n", // http.Error adds a newline
		},
		{
			name:         "Missing User ID",
			userID:       "",
			mockService:  &MockUserService{}, // Mock service not called
			expectedCode: http.StatusBadRequest,
			expectedBody: `User ID is required` + "\n",
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			handler := &UserHandler{Service: tt.mockService}

			req, err := http.NewRequest("GET", "/users?id="+tt.userID, nil)
			if err != nil {
				t.Fatal(err)
			}

			recorder := httptest.NewRecorder()
			handler.GetUser(recorder, req)

			if recorder.Code != tt.expectedCode {
				t.Errorf("expected status code %d, got %d", tt.expectedCode, recorder.Code)
			}

			if reader := bytes.NewReader([]byte(tt.expectedBody)); reader.Len() > 0 {
				if recorder.Body.String() != tt.expectedBody {
					t.Errorf("expected body %q, got %q", tt.expectedBody, recorder.Body.String())
				}
			}
		})
	}
}

func TestUserHandler_CreateUser(t *testing.T) {
	tests := []struct {
		name         string
		requestBody  string
		mockService  *MockUserService
		expectedCode int
		expectedBody string
	}{
		{
			name:        "Successful Create",
			requestBody: `{"Name":"Bob"}`,
			mockService: &MockUserService{
				CreateUserFunc: func(name string) (*User, error) {
					return &User{ID: "new-id", Name: name}, nil
				},
			},
			expectedCode: http.StatusCreated,
			expectedBody: `{"ID":"new-id","Name":"Bob"}`,
		},
		{
			name:        "Invalid Request Body",
			requestBody: `{Invalid JSON}`,
			mockService: &MockUserService{}, // Mock service not called
			expectedCode: http.StatusBadRequest,
			expectedBody: `Invalid request body` + "\n",
		},
		{
			name:        "Service Failure",
			requestBody: `{"Name":"Charlie"}`,
			mockService: &MockUserService{
				CreateUserFunc: func(name string) (*User, error) {
					return nil, errors.New("database error")
				},
			},
			expectedCode: http.StatusInternalServerError,
			expectedBody: `Failed to create user` + "\n",
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			handler := &UserHandler{Service: tt.mockService}

			req, err := http.NewRequest("POST", "/users", bytes.NewBufferString(tt.requestBody))
			if err != nil {
				t.Fatal(err)
			}
			req.Header.Set("Content-Type", "application/json")

			recorder := httptest.NewRecorder()
			handler.CreateUser(recorder, req)

			if recorder.Code != tt.expectedCode {
				t.Errorf("expected status code %d, got %d", tt.expectedCode, recorder.Code)
			}

			if reader := bytes.NewReader([]byte(tt.expectedBody)); reader.Len() > 0 {
				actualBody := bytes.TrimSpace(recorder.Body.Bytes()) // Remove potential trailing newline from http.Error
				expectedBody := bytes.TrimSpace([]byte(tt.expectedBody))

				if !bytes.Equal(actualBody, expectedBody) {
					t.Errorf("expected body %q, got %q", string(expectedBody), string(actualBody))
				}
			}
		})
	}
}

This example demonstrates:

• Using httptest.NewRecorder and http.NewRequest to simulate HTTP requests and capture responses without starting a real HTTP server.
• Implementing a MockUserService to control the behavior of the UserService dependency, allowing us to test UserHandler in isolation.
• Table-driven tests (t.Run) for a clean and comprehensive test suite.

Integration Testing with Docker

While unit tests are crucial, they don't cover the full picture. Real-world applications interact with databases, message queues, and external APIs. Integration tests bridge this gap by testing these interactions. Using Docker greatly simplifies the setup and teardown of these external dependencies, making integration tests reliable and reproducible.

Let's assume our UserService implementation uses a PostgreSQL database.

// real_user_service.go
package main

import (
	"database/sql"
	"fmt"
	"log"

	_ "github.com/lib/pq" // PostgreSQL driver
)

// DBUserService implements UserService using a PostgreSQL database
type DBUserService struct {
	DB *sql.DB
}

func NewDBUserService(dataSourceName string) (*DBUserService, error) {
	db, err := sql.Open("postgres", dataSourceName)
	if err != nil {
		return nil, fmt.Errorf("failed to open database: %w", err)
	}
	if err = db.Ping(); err != nil {
		return nil, fmt.Errorf("failed to connect to database: %w", err)
	}
	log.Println("Successfully connected to PostgreSQL")
	return &DBUserService{DB: db}, nil
}

func (s *DBUserService) GetUserByID(id string) (*User, error) {
	row := s.DB.QueryRow("SELECT id, name FROM users WHERE id = $1", id)
	user := &User{}
	err := row.Scan(&user.ID, &user.Name)
	if err == sql.ErrNoRows {
		return nil, errors.New("user not found")
	}
	if err != nil {
		return nil, fmt.Errorf("failed to get user: %w", err)
	}
	return user, nil
}

func (s *DBUserService) CreateUser(name string) (*User, error) {
	user := &User{Name: name}
	err := s.DB.QueryRow("INSERT INTO users (name) VALUES ($1) RETURNING id", name).Scan(&user.ID)
	if err != nil {
		return nil, fmt.Errorf("failed to create user: %w", err)
	}
	return user, nil
}

// InitSchema creates the users table if it doesn't exist
func (s *DBUserService) InitSchema() error {
	const schema = `
	CREATE TABLE IF NOT EXISTS users (
		id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
		name TEXT NOT NULL
	);`
	_, err := s.DB.Exec(schema)
	return err
}

Docker Compose for Test Dependencies

For integration tests, we'll use Docker Compose to spin up a dedicated PostgreSQL instance. Create a docker-compose.test.yml file:

# docker-compose.test.yml
version: '3.8'
services:
  db_test:
    image: postgres:13
    environment:
      POSTGRES_DB: test_db
      POSTGRES_USER: test_user
      POSTGRES_PASSWORD: test_password
    ports:
      - "5433:5432" # Map to a different port to avoid conflicts with local dev DB
    volumes:
      - pg_data_test:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U test_user -d test_db"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  pg_data_test:

Writing the Integration Test

Now, let's write an integration test for our UserHandler that uses the real DBUserService and connects to the Dockerized PostgreSQL.

We need a helper function to manage the Docker Compose services.

// integration_test_utils.go
package main

import (
	"fmt"
	"os/exec"
	"time"
)

// StartTestContainers brings up the Docker Compose services
func StartTestContainers() error {
	cmd := exec.Command("docker-compose", "-f", "docker-compose.test.yml", "up", "-d")
	output, err := cmd.CombinedOutput()
	if err != nil {
		return fmt.Errorf("failed to start containers: %s, %w", string(output), err)
	}
	fmt.Println("Docker containers started.")

	// Wait for the database to be healthy
	healthCheckCmd := exec.Command("docker-compose", "-f", "docker-compose.test.yml", "ps", "-q", "db_test")
	containerIDBytes, err := healthCheckCmd.CombinedOutput()
	if err != nil {
		return fmt.Errorf("failed to get db_test container ID: %s, %w", string(containerIDBytes), err)
	}
	containerID := string(healthCheckCmd.Output()) // Trim newline

	fmt.Println("Waiting for db_test to be healthy...")
	for i := 0; i < 60; i++ { // Wait up to 5 minutes (60 * 5s interval)
		healthCmd := exec.Command("docker", "inspect", "-f", "{{.State.Health.Status}}", containerID)
		healthOutput, err := healthCmd.CombinedOutput()
		if err == nil && string(healthOutput) == "healthy\n" {
			fmt.Println("db_test is healthy.")
			return nil
		}
		time.Sleep(5 * time.Second)
	}
	return errors.New("db_test health check failed after timeout")
}

// StopTestContainers brings down the Docker Compose services
func StopTestContainers() error {
	cmd := exec.Command("docker-compose", "-f", "docker-compose.test.yml", "down", "-v") // -v removes volumes
	output, err := cmd.CombinedOutput()
	if err != nil {
		return fmt.Errorf("failed to stop containers: %s, %w", string(output), err)
	}
	fmt.Println("Docker containers stopped.")
	return nil
}

And then the integration test itself:

// handlers_integration_test.go
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"net/http/httptest"
	"os"
	"strings"
	"testing"
)

const (
	testDSN = "host=localhost port=5433 user=test_user password=test_password dbname=test_db sslmode=disable"
)

func TestMain(m *testing.M) {
	if err := StartTestContainers(); err != nil {
		fmt.Printf("Failed to start test containers: %v\n", err)
		os.Exit(1)
	}

	// Run tests
	code := m.Run()

	if err := StopTestContainers(); err != nil {
		fmt.Printf("Failed to stop test containers: %v\n", err)
		// Don't exit here, as tests might have passed. Just log the error.
	}

	os.Exit(code)
}

func TestUserHandler_Integration(t *testing.T) {
	// Initialize real DBUserService
	userService, err := NewDBUserService(testDSN)
	if err != nil {
		t.Fatalf("Failed to initialize DBUserService: %v", err)
	}
	defer userService.DB.Close()

	// Ensure schema is clean for each test function run
	if _, err := userService.DB.Exec("DROP TABLE IF EXISTS users;"); err != nil {
		t.Fatalf("Failed to drop existing users table: %v", err)
	}
	if err := userService.InitSchema(); err != nil {
		t.Fatalf("Failed to initialize schema: %v", err)
	}

	handler := &UserHandler{Service: userService}

	t.Run("Create a user successfully", func(t *testing.T) {
		requestBody := `{"Name":"Integration Test User"}`
		req, err := http.NewRequest("POST", "/users", bytes.NewBufferString(requestBody))
		if err != nil {
			t.Fatal(err)
		}
		req.Header.Set("Content-Type", "application/json")

		recorder := httptest.NewRecorder()
		handler.CreateUser(recorder, req)

		if recorder.Code != http.StatusCreated {
			t.Errorf("expected status code %d, got %d. Body: %q", http.StatusCreated, recorder.Code, recorder.Body.String())
		}

		var createdUser User
		err = json.Unmarshal(recorder.Body.Bytes(), &createdUser)
		if err != nil {
			t.Fatalf("Failed to unmarshal response: %v", err)
		}

		if createdUser.ID == "" {
			t.Error("Expected a user ID, got empty")
		}
		if createdUser.Name != "Integration Test User" {
			t.Errorf("Expected name 'Integration Test User', got '%s'", createdUser.Name)
		}
	})

	t.Run("Get a created user successfully", func(t *testing.T) {
		// First, create a user directly via the service or a previous integration step
		testUser := &User{Name: "Another Integration User"}
		err := userService.DB.QueryRow("INSERT INTO users (name) VALUES ($1) RETURNING id", testUser.Name).Scan(&testUser.ID)
		if err != nil {
			t.Fatalf("Failed to pre-create user for GET test: %v", err)
		}

		req, err := http.NewRequest("GET", "/users?id="+testUser.ID, nil)
		if err != nil {
			t.Fatal(err)
		}

		recorder := httptest.NewRecorder()
		handler.GetUser(recorder, req)

		if recorder.Code != http.StatusOK {
			t.Errorf("expected status code %d, got %d. Body: %q", http.StatusOK, recorder.Code, recorder.Body.String())
		}

		var fetchedUser User
		err = json.Unmarshal(recorder.Body.Bytes(), &fetchedUser)
		if err != nil {
			t.Fatalf("Failed to unmarshal response: %v", err)
		}

		if fetchedUser.ID != testUser.ID {
			t.Errorf("Expected user ID %s, got %s", testUser.ID, fetchedUser.ID)
		}
		if fetchedUser.Name != testUser.Name {
			t.Errorf("Expected name %s, got %s", testUser.Name, fetchedUser.Name)
		}
	})

	t.Run("Get a non-existent user", func(t *testing.T) {
		req, err := http.NewRequest("GET", "/users?id=non-existent", nil)
		if err != nil {
			t.Fatal(err)
		}

		recorder := httptest.NewRecorder()
		handler.GetUser(recorder, req)

		if recorder.Code != http.StatusNotFound {
			t.Errorf("expected status code %d, got %d", http.StatusNotFound, recorder.Code)
		}
		expectedBody := "User not found\n"
		if recorder.Body.String() != expectedBody {
			t.Errorf("expected body %q, got %q", expectedBody, recorder.Body.String())
		}
	})
}

To run these integration tests:

1. Make sure Docker is running.
2. Navigate to your project directory.
3. Run go test -v -run Integration ./... (or specifically go test -v -run Integration handlers_integration_test.go).

Explanation:

• The TestMain function in handlers_integration_test.go is special. It runs before any test functions in the package. We use it to orchestrate starting and stopping our Docker Compose services.
• StartTestContainers uses docker-compose up -d to bring up the services in the background and includes a health check loop to wait for the database readiness.
• StopTestContainers uses docker-compose down -v to tear down the services and remove associated volumes, ensuring a clean state for subsequent test runs.
• Inside TestUserHandler_Integration, we initialize a real DBUserService connected to our Dockerized PostgreSQL.
• Importantly, for each test run or test case, we ensure the database schema is reset (DROP TABLE IF EXISTS users; and InitSchema()). This provides test isolation, preventing one test from affecting another.
• We use httptest.NewRecorder and http.NewRequest again, but this time, the UserHandler interacts with a real database instead of a mock.

Conclusion

A well-structured testing strategy is paramount for developing reliable Go web applications. By mastering unit testing with Go's standard library and carefully employing mocks for dependency isolation, you build a fast and stable foundation. Extending this with Docker-powered integration tests allows you to verify the full stack, including interactions with real external services, ensuring your application behaves correctly in a production-like environment. This layered approach to testing, from the smallest unit to the complete system, ultimately leads to more robust, maintainable, and trustworthy web applications.