As mentioned in the previous post, having a set of metrics and tools to measure the quality of a given verification environment is really critical. The metric should not only measure the quality of the tests but also the capability to capture any abnormal behavior. Before sharing how people were trying to do this, we would like to share some practical experiences that might be encountered on the road.

One of the most challenging jobs of promoting any metric is actually not from the technical side. Instead, it is more likely from the political/business side. Big companies usually refuse to adopt proprietary methods, formats, or metrics to make sure they won’t be tethered to one single vendor. However, providing innovative solutions usually means having to use supplicated tools which only one or a few vendors can provide. This in turn usually means that the single vendor (or small group of vendors) will do their best to avoid competition. Therefore, a proprietary language or method will show up.

In the end, the proposed metric makes a lot of sense. The “proprietary” factor could make it difficult to go through the deployment process. This situation is worse between IP vendors compared to internal IP providers. Each IP vendor is an individual company. Trying to ask them to adopt some proprietary metric/tool together could be really tough.

The solution? Some people suggest that metric providers (aka startups) should create a fake company to work on the similar thing. In this way, the big companies won’t worry about being tied to one vendor Joking aside… No, we don’t know the answer yet but that’s the fun part of the game…