Today's intelligent devices contain increasingly complex architectures, often running millions of lines of code. Development teams are increasingly squeezed on both time and costs. Software quality assurance has never been more important—or more difficult.
To realize market potential while controlling costs and mitigating risks, device manufacturers need more automation tools that help them monitor, manage, and optimize the testing process. Designed to meet these challenges, Wind River Test Management is a collaborative automation solution for embedded device system test and diagnostics.
Wind River Test Management leverages unique, dynamic instrumentation technology to measure code coverage, profile performance, enable white-box testing, and diagnose and repair the system, all at run-time. The system is designed to maximize resource utilization and accelerate the device software testing process:
Overview
Managers
Testers
Developers
Run-Time
Architecture
A Collaborative Automation Solution
Wind River Test Management provides an integrated suite of applications built on a collaboration platform and test information repository that can be shared by quality assurance (QA) personnel and development engineers.
- Capture and manage test plans
- Develop reusable manual and scripted test cases
- Automate system test execution across multiple devices
- Manage virtual labs of test devices, remotely
- Monitor code coverage at run-time
- Profile device performance at run-time
- Incorporate run-time white-box testing techniques
- Collaborate to diagnose run-time defects
- Patch software "hitlessly" without stopping devices
- Manage with interactive dashboards
Benefits for Managers
Get Higher-Quality Devices to Market Faster at Less Cost
Wind River Test Management helps teams adopt a repeatable process for rigorously testing embedded software, detecting bugs earlier, and correcting faults faster. It helps break down the wall between developers and QA staff by providing a common automation solution that fosters collaboration and knowledge sharing. The system streamlines workflow, automates testing, and mitigates risks and delays so products can get to market faster at lower cost.
Test More Software More Often
Wind River Test Management automates the key test life cycle functions for better, deeper system testing. It captures manual and scripted tests within a management environment that fosters testing "early and often." The system enables new white-box testing techniques to more fully exercise and validate hard-to-achieve device states, expanding code coverage and improving quality.
Dramatically Accelerate Defect Resolution
Wind River Test Management eliminates the delays caused by hit-or-miss fault isolation and long rebuild times. QA teams can dynamically deploy diagnostics probes to gather run-time information without stopping lab devices. Developers can quickly identify, debug, and dynamically patch defects, dramatically accelerating the defect resolution cycle.
Manage Progress, Quality, and Resources
Managers utilize a Web browser–based dashboard to track progress of the testing effort. They can review aggregate test results, monitor the rate that tests are written and executed, and get visibility into what device software has actually been tested along with the test results and quality levels. The system also enables the management and scheduling of lab devices, helping prevent team inefficiencies and avoiding underutilized capital equipment.
Wind River Deployment Services
With solutions in more than 350 million deployed devices, Wind River has a 25-year track record of innovation and reliability. Wind River Professional Services, a CMMI Level 3–certified organization, can offer best practice deployment services or custom-configure Wind River Test Management to adapt to your processes and device platform needs. Our world-class support organization stands ready to handle your technical inquiries.
Benefits for Testers
Manage Test Plans and Schedules
Capturing and organizing tests can be a problem when using manual means based on word processing documents or spreadsheets. Tracking test development is equally challenging, especially when test teams are spread around the world. Wind River Test Management provides a test planning application that lets you construct a hierarchical view of your test plan and specify the test cases you need, assign these to test developers, and track progress of the test case development effort.
Develop Reusable Test Cases
Wind River Test Management supports creating manual tests and allows scripts to be imported, stored, and organized within the system as automated test cases. The product supports any scripting language, such as TCL, PERL, and Python, or can use compiled executable tests.
Manage Virtual Labs of Test Devices
Many companies have expensive capital equipment spread across many labs in their company. Wind River Test Management provides a virtual lab management application that lets users define an appropriate virtual group of device assets and remotely manage, provision, and access these devices for automated testing. This saves time and hassle and allows maximum utilization of people and capital equipment resources.
Automate Test Execution
Wind River Test Management provides an easy-to-use test bed for executing large numbers of tests automatically on devices and then analyzing their results. The test planner sets up one or more test suites—collections of test cases—and assigns testers to run the suites on a reserved virtual lab device. The current software build can be remotely loaded on the device, and test results are captured by the system for viewing via Web browsers. Actions can also be triggered automatically via an open API that allows, for example, a test suite to be run nightly after a new build is complete.
Analyze the Device at Run-Time
Wind River Test Management provides testers the ability to use dynamically applied instrumentation to gather run-time code coverage, performance profiling, and diagnostics data. This same technology can be used to extend tests with white-box access to device internals at run-time.
Benefits for Developers
Diagnose Defects Dynamically
Isolating and resolving defects in embedded devices is typically a long process that requires the collaboration of both testers and developers. Traditional debug tools are too intrusive to be deployed in system test labs, so Wind River Test Management allows developers to create dynamic instrumentation to remotely probe within a lab device to gather "forensic" data at run-time. Device software can be instrumented on the fly (e.g., with printf or log statements), without rebuilding or rebooting the device, dramatically decreasing the cycle time required to isolate defects.
Patch Software on the Fly
When developers isolate and fix defects at their desks, they need to validate the fix in the lab under test. This can be a time-consuming process. Wind River Test Management provides a dynamic patching mechanism where developers can publish a fix to the test group who can then inject it into the lab device—without a full recompile, rebuild, and reload, and without stopping the device—saving time and money.
Collaborate Across the World
Many development and test organizations today are in different physical locations and time zones. This can significantly limit test productivity, as developers and testers try to work together to solve tough problems. Wind River Test Management enables remote diagnostic access to managed lab devices and rapid deployment of defect-isolating instrumentation. This lets developers and testers collaborate much more productively around real device data rather than reported symptoms.
Wind River Test Diagnostics
For development teams interested in diagnostics and patching only, Wind River packages a subset of Wind River Test Management called Wind River Test Diagnostics for use in system diagnostics and integration labs.
Unique Run-Time Features
Monitor Code Coverage at Run-Time
It's often difficult for test managers to measure the quality of their test plan, optimize test execution, or know what device software has been tested. Wind River Test Management can detect and record specifically which software functions within a build were tested or not tested. This run-time code coverage capability is an on-demand feature that can be enabled and disabled by testers for specific modules and test runs, letting testers and their managers know how much of their device is actually being tested.
Profile Device Performance at Run-Time
Often testers are required to measure performance of devices under test to validate that execution times meet specifications or to help isolate bottlenecks. Wind River Test Management provides a unique way to profile run-time device performance. The system allows testers to dynamically add performance measurement probes on any function within the device software, on-demand, with minimal footprint, and without a software rebuild.
Incorporate White-Box Testing
Most device testing, whether manual or scripted, is typically "black box" testing. This is effective for certain functional testing but it can be difficult to get the device into a desired state or to validate that unusual states such as error conditions are handled correctly. Wind River Test Management provides unique white-box testing features through use of sensorpoint dynamic instrumentation. Testers or their developer counterparts can create reusable sensorpoint probes that can be dynamically injected into the device before a test run to input data, force state, read or log data at a given subsystem, or inject faults or failures.
Diagnose Defects Dynamically
An inevitable outcome of testing is the discovery of defects. Isolating and resolving defects in embedded devices requires the collaboration of both testers and developers. Wind River Test Management allows developers and testers to collaborate to gather run-time "forensic" data about device defects to rapidly get to root cause.
Powerful, Scalable Architecture
Wind River Sensorpoint Technology
Wind River's sensorpoint technology lets users probe or patch running devices without rebuilding the device software or stopping the device. Sensorpoints allow developers and testers to inject code into a running executable to gather data, force faults, add functionality, diagnose problems, gather run-time analytics information, or do white-box testing. Sensorpoints are minimally intrusive on device performance and footprint so they can be used in actual devices under test.
Open API
Wind River Test Management consists of a scalable, distributed architecture that brings together the development and testing players into a common environment. The system comprises Web browser-based applications, a relational database, a virtual lab manager, command-line and Eclipse-based developer tools, a scalable server infrastructure, and downloadable device agents. An open server application programmers interface (API), based on standard Web Services technology, allows integration with third-party applications or external automation scripts.
Broad Platform Support
Wind River Test Management was designed for embedded device system test and aims to support the widest range of device processor, operating system, host, and infrastructure software combinations with the product. This list is continuously expanding to cover Wind River Linux, VxWorks, and non–Wind River platforms. Contact your Wind River representative for details.
