Tools And Technologies Used
This project leverages a modern tech stack combining backend development, data storage, visualization, and containerization to streamline test result tracking and reporting.
Java: The core programming language used to build the backend logic of the application. Its versatility and robustness make it ideal for scalable applications.
PostgreSQL: A powerful open-source relational database used to store test execution data efficiently and reliably.
Grafana: A data visualization tool that turns raw test metrics into insightful and interactive dashboards for easier analysis.
Docker: Simplifies the deployment process by containerizing the entire application environment, including PostgreSQL and Grafana.
Gradle: A build automation tool used for compiling code, managing dependencies, and packaging the application.
Spring Boot: Provides a production-ready framework to build stand-alone Java applications with minimal configuration.
Spring Boot Data JPA: Simplifies database interactions by using Java Persistence API (JPA) for seamless CRUD operations on PostgreSQL.
Dashboards
Dashboards provide a visual and intuitive way to monitor the health, performance, and reliability of your tests across various dimensions. Powered by Grafana, these dashboards turn raw test data into actionable insights, making it easier to spot issues, trends, and opportunities for improvement. Here are the key dashboards available in the project:
Service Level Test Summary
This dashboard presents the service-level test summary over a time series, making it easier to visualize the number of test cases that passed, failed, or skipped with each build. It also displays the current coverage, the total number of tests, and the count of successful and failed builds. This dashboard is mostly used for unit test cases since we have automated coverage for them but can also be used for end to end testcases if we can define some coverage or otherwise keep coverage 0.
All build is a simple table that shows the test_type, service, environment, total_executed, total_passed, total_failed, and total skipped test cases.
This table displays each test case along with its most recent status and the number of times it has been executed. It’s an important metric for identifying flaky test cases.
Traceability
The traceability dashboard shows the mapping of a test case to its story or feature and the test case's recent status.
Quick Local Setup
Setting up the test-metrics project locally is simple and efficient, thanks to Docker. This section provides step-by-step instructions to get PostgreSQL and Grafana running in isolated containers, configure your environment, and launch the Spring Boot application. Whether you're testing features or contributing to development, this guide will help you get everything up and running in minutes.
Install Docker
Docker is a platform used to develop, ship, and run applications inside containers. It simplifies setting up environments by isolating dependencies and system configurations.
Run the following Docker command to start a PostgreSQL container within a custom network (test_network). This sets up your database for storing test execution results: - docker run --network test_network --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=treco -e POSTGRES_USER=postgres -d postgres
This command:
Creates a container named postgres
Maps local port 5432 to the container’s port
Sets up database credentials and a default DB called treco
Start Grafana Using Docker
- docker run -d -p 3000:3000 --network test_network --name=grafana --volume grafana-storage:/var/lib/grafana grafana/grafana-enterprise
This starts Grafana on port 3000 and connects it to the same Docker network (test_network) as the PostgreSQL container, allowing seamless communication.
Add PostgreSQL as a Data Source in Grafana
After Grafana is running:
Log in to Grafana (default user: admin, password: admin)
Navigate to Configuration → Data Sources
Add PostgreSQL as a new data source
Use the Docker network IP of the PostgreSQL container (instead of localhost) and the default port 5432
Provide the same credentials used during container setup
This enables Grafana to query data stored in PostgreSQL.
Add Dashboards in Grafana
Import pre-configured dashboards by uploading the JSON files available in the GitHub repository. These dashboards display test execution trends, failures, and other metrics in an interactive, visual format.
Start the Web Application Using IntelliJ or Gradle
You can run the Spring Boot application using IntelliJ IDE or via the Gradle command line. This application exposes an endpoint:
This endpoint accepts test execution reports (in JUnit XML format) and publishes the parsed data to the PostgreSQL database, making it available for visualization in Grafana.
Visit the GitHub repository for more information on the 'test-metrics' project and to explore its capabilities. Whether you're a seasoned developer or just starting out, 'test-metrics' is a valuable addition to your software development toolkit. It’s designed to be simple, extensible, and developer-friendly, making it easy to integrate into any workflow. This project is open to feedback, suggestions, and contributions from the community—your input can help shape its future and make it even better for everyone involved. Don’t hesitate to fork the repo, raise issues, or submit pull requests!
