Metrics and Measurement
(Unit IV)
- A key element of any engineering process is measurement.
- Using measures, we can assess the quality of the engineered products or systems that we build.
- Within the software engineering context, a measure provides a quantitative indication of the extent, amount, dimension, capacity, or size of some attribute of product or a process.
- Measurement is the act of determining a measure. It is the process by which numbers or symbols are assigned to the attributes of entities is the real world in such a way as to define them according to clearly defined rules.
Product Metrics:
- IEEE defines metrics as "a quantitative measure of the degree to which a system, component, or process possesses a given attribute"
- By its nature, Software Engineering is a quantifiable approach. In this approach product metrics help Software Engineers gain insight into the design and construction of the software they build by focusing on specific, measurable attributes of software engineering work products.
- Product metrics provide a systematic way to assess the quality of software based on a set of clearly defined rules.
Indicators:
- An indicator is a metric or combination of metrics that provides insight into the software process, a software project or the product itself.
- It enables the project manager or software engineer to adjust the process, the project, or the product to make things better.
Quality and Efficiency:
- Quality and Efficiency are considered indirect because they are external attributes of a product and cannot be measured directly by metrics like lines of code or cost, which are direct measures.
- Quality is indirect measure that encompasses many other attributes, such as reliability, usability, and security
- Efficiency is also an indirect measure that assesses how well a software product performs its function, like its response time or use of resources.
Software Metrics
- Software quality metrics are quantifiable measures used to assess the characteristics of software products, the effectiveness of the development process, and the progress of a software project.
- These metrics are often categorized into three main types: Product, Process, and Project metrics.
Product Metrics:
These metrics evaluate the inherent quality characteristics of the software itself, focusing on aspects visible to the end-user or related to the codebase's structural integrity.
| Metric | Description | Example/Measurement |
| Defect Density | The number of confirmed defects (bugs) found in a specific size of the software. | Defects per Thousand Lines of Code (KLOC) or per Function Point. |
| Reliability Metrics | Measures the software's ability to perform its required functions under stated conditions for a specified period of time. | Mean Time Between Failures (MTBF): Average time the system operates without failure. Mean Time To Failure (MTTF): Average time until the first failure. |
| Maintainability | The ease with which the software can be modified to correct defects, improve performance, or adapt to a changing environment. | Mean Time To Change (MTTC): Average time taken to implement a change. |
| Test Coverage | The percentage of the source code executed by test cases. | Percentage of code lines, branches, or paths covered by tests. |
| Performance | Measures the system's efficiency in terms of responsiveness, throughput, and resource utilization. | Response time, transaction rate, resource usage (CPU/Memory). |
| Security | Measures the software's ability to protect against unauthorized access, modification, or destruction. | Number of known security vulnerabilities per module. |
| Code Quality/Complexity | Measures how well-written, readable, and non-redundant the code is. | Cyclomatic Complexity: Measures the number of independent paths through a program's source code. |
| Customer Satisfaction | Gauges how well the product meets user needs and expectations. | Net Promoter Score (NPS), Customer Satisfaction Score (CSAT), or crash rate. |
Process Metrics:
These metrics focus on the effectiveness and efficiency of the development, testing, and maintenance processes.
| Metric | Description | Example/Measurement |
| Defect Removal Efficiency (DRE) | A measure of the development team's ability to remove defects before the software reaches the end-user. | DRE = Defects Found Before Release / Total Defects Found |
| Defect Leakage | The percentage of defects that escape the testing process and are found by customers in production. | Leakage Rate = Defects Found in Production / (Defects Found in Testing + Production) |
| Mean Time to Recovery (MTTR) | The average time required to restore the system to full functionality after a failure or incident. | Time from incident start to full service restoration. |
| Change Failure Rate | The percentage of changes (e.g., software releases) to productioWhich Testing is the re-execution of some subsets of tests that have already been conducted to ensure the changes that are not propagated? 1) Unit Testing 2) Regression Testing 3) Integration Testing 4) Thread-based Testingn that result in a degraded service or require remediation. | Number of failed deployments / Total number of deployments. |
| Test Case Pass Rate | The percentage of executed test cases that pass successfully. | Number of Passed Test Cases / Total Number of Test Cases Executed |
Project Metrics:
These metrics track project characteristics and execution, often related to resources, timelines, and costs, which indirectly affect quality.
| Metric | Description | Example/Measurement |
| Lead Time for Changes | The time it takes for a code change to go from initial commit to successful deployment in a production environment. | Time elapsed from code commit to deployment. |
| Deployment Frequency | How often a team successfully releases to production. | Deployments per day, week, or month. |
| Cost of Quality (CoQ) | The total investment in achieving and maintaining product quality, including prevention, appraisal, and failure costs. | Sum of costs for testing, quality assurance, bug fixing, and downtime. |
| Schedule/Effort Variance | The difference between the planned and actual schedule or effort for project tasks. | Percentage deviation from the planned schedule or budget. |
- The choice of which metrics to track depends heavily on the project goals, the development methodology (like Agile or Waterfall), and the critical nature of the software.
- A balanced approach using a combination of these metrics provides the most comprehensive view of software quality.
