It is clear that functional verification is a major problem today and that we need better solutions. But what are the characteristics of a better solution?
Reduced resources and schedule
The most obvious characteristic of a better solution is that it should require less people and should enable a shorter schedule. The fully loaded cost of verification engineering time ranges from $100K/yr for junior engineers to upwards of $200K/yr for highly experienced senior engineers. With an average team size of 38 people1 and the average project duration prior to tape-release of nearly 11 months1, the cost of verification in people alone can range in the millions of dollars. Since verification is very often on the critical path of the project, the verification process can have a significant impact on the schedule. A late project, due to pre-silicon verification slips or post-silicon functional bugs, can have a multi-million dollar impact on the company's bottom line. In cases where the market space is highly competitive, a late project can even adversely impact the company’s long-term viability.
Better predictability
Another characteristic of a better solution is that it should enable managers to better quantify risk. At the end of the day, verification can never be 100% certain. Instead, design managers implicitly or explicitly assess the level of confidence in the verification process. When the costs of verification outweigh the risk of bugs (i.e. delay of schedule, cost or re-spins, revenue loss, customer satisfaction, etc) , the design manager can sign off on a tape-release. To make this call, design managers and their teams need metrics to help them assess confidence in the verification process. The figure on the right shows the impact that poor predictability can have on both schedule and quality. In this figure, the lower line represents the cumulative distribution of bugs for a project using a verification methodology with poor predictability. The upper line represents the cumulative distribution of bugs for a project using a verification methodology with better predictability. At weeks 15, 19, and 26, a design manager might infer that the verification process is converging, because the number of bugs at those times appears to be stabilizing. Based on the bugs found in succeeding weeks, however, those inferences would have been wrong. Finally, at week 34, the manager might infer that the verification process has finally converged, and therefore sign off on the design. However, based on the actual number of bugs in the design, this would have been a poor decision and would have resulted in a poor quality tape-release.
Ease of adoption
A solution that is better in resources, schedule, and/or predictability, but that cannot be adopted by the team is not a feasible solution. To be easily adoptable, a solution should:
- Utilize engineer skill-sets which are generally available or easily acquired
- Easily integrate into existing methodologies
- Allow for incremental adoption so that incremental increases in effort yield corresponding improvements in schedule, resources, or predictability
Read a high-level overview of Hyper Analytix's solution and how it addresses the areas of schedule and resources, predictability, and ease of adoption
1 Collett International, Design Closure Study, December 2005