ADR-004: Testing Strategy for Quality Assurance

Testing Strategy: Quality Check for Future Additions

A focused testing approach that serves as both quality assurance for future changes and documentation of user behavior expectations.

1. History (The Problem Context)

The Problem

Due to lack of testing in sensitive parts of the code such as business logic and widgets, we often found when adding new features to existing projects that we were breaking previously perfectly functioning code.

Context

• When: Discovered during feature additions to existing projects
• Where: Business logic in services, UI state management in notifiers, widget behavior
• Who: Development team experiencing regression issues
• Constraints:
  • Limited time for comprehensive testing
  • Need to focus on most critical areas
  • Must prevent regressions in working functionality
  • Testing should serve as behavior documentation

Previous Approach

• Minimal or no testing in critical areas
• Manual testing only
• Discovering breaks after deployment
• No documentation of expected behavior

// Previous approach - no tests
class FeatureService {
  Future<List<Item>> getItems() {
    // Complex business logic with no tests
    // Changes here could break existing functionality
  }
}

2. Reason (Architecture Decision Record)

Decision

Implement focused testing strategy covering three critical areas:

1. Application Services - All business logic and cross-cutting concerns
2. Notifiers - UI state management and user interactions
3. Individual Widgets - Component behavior and user interface

Alternatives Considered

1. Comprehensive Testing (Everything)

   • Pros: Complete coverage, maximum confidence
   • Cons: Too time-consuming, diminishing returns on some areas

2. Integration Testing Only

   • Pros: Tests real user flows
   • Cons: Slow feedback, hard to isolate issues, doesn’t document component behavior

3. No Testing (Status Quo)

   • Pros: Fastest development initially
   • Cons: Regressions, no behavior documentation, fear of changing code

4. Focused Testing (Chosen) ✅

   • Pros: Covers most critical areas, reasonable time investment, prevents regressions
   • Cons: Some areas uncovered, requires discipline to maintain

Rationale

Testing serves two critical purposes:

1. Quality Check for Future Additions: Prevents breaking existing functionality
2. User Behavior Documentation: Tests document expected behavior and interactions

Focusing on services, notifiers, and widgets covers the areas where:

• Business logic lives (services)
• State management happens (notifiers)
• User interactions occur (widgets)

3. Implementation (Code Snippets & Walkthrough)

Testing Framework: flutter_test

Always use flutter_test package (not just test) for all testing:

pubspec.yaml

dev_dependencies:
  flutter_test:
    sdk: flutter
  mocktail: ^1.0.0

Step 1: Application Services Testing

Test all business logic and cross-cutting concerns:

test/unit/services/feature_service_test.dart

import 'package:flutter_test/flutter_test.dart';
import 'package:mocktail/mocktail.dart';

class MockFeatureRepository extends Mock implements IFeatureRepository {}

void main() {
  group('FeatureService', () {
    late FeatureService service;
    late MockFeatureRepository mockRepository;

    setUp(() {
      mockRepository = MockFeatureRepository();
      service = FeatureService(repository: mockRepository);
    });

    group('getItems', () {
      test('returns items when repository succeeds', () async {
        // Arrange
        final expectedItems = [FeatureItem(id: '1', name: 'Test')];
        when(() => mockRepository.getItems()).thenAnswer((_) async => expectedItems);

        // Act
        final result = await service.getItems();

        // Assert
        expect(result, equals(expectedItems));
        verify(() => mockRepository.getItems()).called(1);
      });

      test('throws domain exception when business rule violated', () async {
        // Arrange
        when(() => mockRepository.getItems()).thenThrow(InvalidStateException('Invalid state'));

        // Act & Assert
        expect(() => service.getItems(), throwsA(isA<InvalidStateException>()));
      });
    });

    group('createItem', () {
      test('validates input and creates item successfully', () async {
        // Test business logic validation
        final input = CreateFeatureInput(name: 'Valid Name');
        final expectedItem = FeatureItem(id: '1', name: 'Valid Name');

        when(() => mockRepository.create(any())).thenAnswer((_) async => expectedItem);

        final result = await service.createItem(input);

        expect(result, equals(expectedItem));
        verify(() => mockRepository.create(input)).called(1);
      });

      test('throws validation exception for invalid input', () async {
        // Test business rule enforcement
        final invalidInput = CreateFeatureInput(name: ''); // Empty name

        expect(() => service.createItem(invalidInput), throwsA(isA<ValidationException>()));
        verifyNever(() => mockRepository.create(any()));
      });
    });
  });
}

Step 2: Notifiers Testing

Test UI state management and user interactions:

test/unit/notifiers/feature_notifier_test.dart

import 'package:flutter_test/flutter_test.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:mocktail/mocktail.dart';

class MockFeatureService extends Mock implements IFeatureService {}

void main() {
  group('FeatureNotifier', () {
    late MockFeatureService mockService;
    late ProviderContainer container;

    setUp(() {
      mockService = MockFeatureService();
      container = ProviderContainer(
        overrides: [
          featureServiceProvider.overrideWith((ref) => mockService),
        ],
      );
    });

    tearDown(() {
      container.dispose();
    });

    test('initial state is loading', () {
      final notifier = container.read(featureNotifierProvider.notifier);
      final state = container.read(featureNotifierProvider);

      expect(state, const AsyncValue<List<FeatureItem>>.loading());
    });

    test('loadItems updates state to data when service succeeds', () async {
      // Arrange
      final items = [FeatureItem(id: '1', name: 'Test')];
      when(() => mockService.getItems()).thenAnswer((_) async => items);

      final notifier = container.read(featureNotifierProvider.notifier);

      // Act
      await notifier.loadItems();

      // Assert
      final state = container.read(featureNotifierProvider);
      expect(state, AsyncValue.data(items));
      verify(() => mockService.getItems()).called(1);
    });

    test('loadItems updates state to error when service fails', () async {
      // Arrange
      final error = Exception('Service error');
      when(() => mockService.getItems()).thenThrow(error);

      final notifier = container.read(featureNotifierProvider.notifier);

      // Act
      await notifier.loadItems();

      // Assert
      final state = container.read(featureNotifierProvider);
      expect(state.hasError, true);
      expect(state.error, error);
    });

    test('refresh clears current data and reloads', () async {
      // Test user interaction behavior
      final items = [FeatureItem(id: '1', name: 'Test')];
      when(() => mockService.getItems()).thenAnswer((_) async => items);

      final notifier = container.read(featureNotifierProvider.notifier);

      // Load initial data
      await notifier.loadItems();
      expect(container.read(featureNotifierProvider).hasValue, true);

      // Refresh
      final refreshFuture = notifier.refresh();

      // Should be loading during refresh
      expect(container.read(featureNotifierProvider).isLoading, true);

      await refreshFuture;

      // Should have data again
      expect(container.read(featureNotifierProvider), AsyncValue.data(items));
      verify(() => mockService.getItems()).called(2); // Initial + refresh
    });
  });
}

Step 3: Individual Widgets Testing

Test component behavior and user interface:

test/widget/components/feature_card_test.dart

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  group('FeatureCard', () {
    testWidgets('displays item information correctly', (tester) async {
      // Arrange
      final item = FeatureItem(id: '1', name: 'Test Item', description: 'Test Description');

      // Act
      await tester.pumpWidget(
        MaterialApp(
          home: Scaffold(
            body: FeatureCard(item: item),
          ),
        ),
      );

      // Assert
      expect(find.text('Test Item'), findsOneWidget);
      expect(find.text('Test Description'), findsOneWidget);
    });

    testWidgets('calls onTap when tapped', (tester) async {
      // Arrange
      final item = FeatureItem(id: '1', name: 'Test Item');
      bool wasTapped = false;

      // Act
      await tester.pumpWidget(
        MaterialApp(
          home: Scaffold(
            body: FeatureCard(
              item: item,
              onTap: () => wasTapped = true,
            ),
          ),
        ),
      );

      await tester.tap(find.byType(FeatureCard));

      // Assert
      expect(wasTapped, true);
    });

    testWidgets('shows loading state when item is null', (tester) async {
      // Act
      await tester.pumpWidget(
        MaterialApp(
          home: Scaffold(
            body: FeatureCard(item: null),
          ),
        ),
      );

      // Assert
      expect(find.byType(CircularProgressIndicator), findsOneWidget);
    });

    testWidgets('handles long text gracefully', (tester) async {
      // Test edge case behavior
      final item = FeatureItem(
        id: '1',
        name: 'Very Long Name That Should Be Truncated Properly',
        description: 'Very long description that should wrap or truncate appropriately without causing overflow issues',
      );

      await tester.pumpWidget(
        MaterialApp(
          home: Scaffold(
            body: SizedBox(
              width: 200, // Constrained width
              child: FeatureCard(item: item),
            ),
          ),
        ),
      );

      // Should not have overflow
      expect(tester.takeException(), isNull);
    });
  });
}

Key Patterns to Follow

1. Use flutter_test: Always use flutter_test package, never just test
2. Mock Dependencies: Use mocktail for mocking external dependencies
3. Test Behavior: Focus on testing behavior, not implementation details
4. Document Edge Cases: Tests serve as documentation of expected behavior
5. Arrange-Act-Assert: Clear test structure for readability

Common Pitfalls

• Testing Implementation Details: Test behavior, not internal state
• Overly Complex Tests: Keep tests simple and focused
• Missing Edge Cases: Test error conditions and boundary cases
• Not Using flutter_test: Always use Flutter’s testing framework

4. Impact & Consequences

Positive Consequences

• Prevents Regressions: Catches breaking changes before deployment
• Behavior Documentation: Tests document expected functionality
• Confidence in Changes: Safe to refactor and add features
• Quality Assurance: Ensures business logic works as expected
• User Experience Protection: Widget tests ensure UI behaves correctly

Negative Consequences

• Initial Time Investment: Writing tests takes development time
• Maintenance Overhead: Tests need updates when behavior changes
• Limited Coverage: Only covers three areas, not everything

Monitoring & Success Metrics

• Regression Prevention: Fewer bugs reported in tested areas
• Development Confidence: Team feels safe making changes
• Test Coverage: Maintain coverage in the three focus areas
• Test Execution Time: Keep test suite fast for quick feedback

5. Related Decisions

• ADR-001: Four-Layer Architecture - Testing aligns with layer responsibilities
• ADR-003: GraphQL Datasource Pattern - How to test datasources

6. References

• Flutter Testing Documentation
• Mocktail Package
• Riverpod Testing Guide

7. Amendment: Value-Focused Testing (2026-02)

Evolution

The original ADR established focused testing on three areas (services, notifiers, widgets) using Arrange-Act-Assert. After practice, we identified two improvements:

1. Coverage metrics create perverse incentives. Teams wrote low-value tests to hit percentage targets instead of protecting real business invariants. All percentage-based coverage targets are removed.

2. Two test languages are more effective than one. Business domain code benefits from BDD-style Given/When/Then naming that documents scenarios. Technical components (Convertors, CacheHandlerSpecs) are better served by standard unit tests that describe input/output.

What Changed

• BDD-style Given/When/Then adopted for services, complex notifiers, repository integration, and widget user scenarios
• Standard unit tests retained for convertors, cache handlers, stream convertors, and pure functions
• TestContext classes replace scattered setUp/tearDown for non-trivial test files (>3 tests)
• Table-driven tests adopted for state machines and exhaustive cases
• Behavioral grouping replaces method-per-group organization for business logic tests
• Coverage percentage targets removed from all standards and checklists

Rationale

Coverage metrics measure execution, not correctness. A test suite with 95% line coverage can still miss critical business invariants while testing implementation details that change with every refactor. Value-focused testing asks: “Does this test prove something a user or consumer cares about?”

Updated References

• How-to: Implement Value-Focused Testing — step-by-step guide with the new patterns
• Testing Patterns Reference — quick-reference decision tables and templates

---

Decision Status: Accepted (Amended)

Last Updated: 2026-02 Original: 2024-07-10 Amendment: Value-focused testing with BDD + TestContext patterns