One of the most challenging problems related to embedded Linux systems is the so called post-mortem analysis. Post-mortem is a Latin expression that means "after death.". In this context, death is meant as an event after which the system becomes unstable or even gets stuck. Therefore, post-mortem analysis refers to the tasks carried out after the occurrence of such an event to figure out its root cause. Even worse, post-mortem analyses are yet harder when these events occur randomly and it is apparently impossible to trigger them in a controlled fashion. In spite of thorough testing at qualification stage, unfortunately, these situations may even occur when the system has already been deployed on the field and is used by end customers making the analysis amazingly troublesome.

Even worse, Several techniques are available for post-mortem analyses analysis. Software tools, hardware tools, or a combination of both can be leveraged. This article is the first of a series of Technical Notes (TN) describing in more details some of these techniques. Interestingly, some of such articles refer to real-world cases for which DAVE Embedded Systems deployed its expertise to support customers reporting on-field failures they were unable to analyze with traditional debugging tools and approaches. In these cases, information reported by customers are yet harder when these events occur randomly necessarily so limited and fragmented that often it is apparently even impossible to trigger them in determine a controlled fashionpriori if the root cause is software or hardware related. SometimesThus, these situations occur when no assumption about the system has already been deployed on the field root cause domain can be made and is used by end customers making the analysis amazingly troublesomeengineers need to be very open-minded to consider every possible cause.

Several techniques are available for post-mortem analysis. Software tools, hardware tools, or a combination of both can be leveraged. This article is the first of a == Articles in this series of Technical Notes (TN) describing real-world cases in which DAVE Embedded Systems put in filed its expertise and leveraged some of these techniques to support several customers reporting on-field failures they were able to analyze with traditional debugging tools.==TBD