Linux Applications Debugging Techniques/Appendices

Tools

 * Overview: http://www.brendangregg.com/Perf/linux_observability_tools.png
 * bpftrace

Interrupted calls
A number of API functions return an error code if the call was interrupted by a signal. Usually this is not an error by itself and the call should be restarted. For instance:

The list of interruptible function differs from Unix-like platform to platform. For Linux see signal(7). Note that sem_wait & co. are on list.

Spurious wake-ups
Threads waiting on a pthreads condition variable can be waken up even if the condition hs not been met. Upon waking up, the condition should be explicitly checked and return waiting if it is not met.

Code review checklist

 * 1) Functional impact
 * 2) Which modules are affected; which functionality is impacted.
 * 3) Which modules & functionality is consumed by the changed code.
 * 4) Which modules/functionality is using it.
 * 5) Side effects.
 * 6) Architecture
 * 7) Scalability
 * 8) Stability
 * 9) Functionality
 * 10) Robustness
 * 11) Technicalities
 * 12) Generic
 * 13) First pass: logic. Correctness. What does it (tries to) do.
 * 14) Side effects.
 * 15) Why is code there?
 * 16) Second pass: error handling & odd conditions.
 * 17) Side effects.
 * 18) Are returned codes checked. Can the code handle them.
 * 19) Arguments:
 * 20) Border/out of bounds/strange values handled.
 * 21) Arg type: convenient?
 * 22) Where is the data coming from? Three levels upstream.
 * 23) Where are the outputs going? Three levels downstream.
 * 24) What are the possible/permitted outputs? Are all handled downstream?
 * 25) Variables: initialized?
 * 26) (over/under)flows:
 * 27) buffers.
 * 28) integral types.
 * 29) Asserted invariants & assumptions.
 * 30) Loops: off-by-one? infinite?
 * 31) Recursion: base case?
 * 32) Third pass: thread safety.
 * 33) Localization?
 * 34) Duplicate information?
 * 35) Math:
 * 36) Small values processed first?
 * 37) NaN checks/std::isnan & friends.
 * 38) Division by zero.
 * C
 * 1) return: resource handling.
 * 2) Use the N functions (ex. s'n'printf vs. sprintf).
 * 3) printf & friends: do formatters match type and number of arguments?
 * 4) Non re-entrant/non thread-safe API
 * 5) Unsafe/race condition afflicted API ( mktemp )
 * 6) enums:
 * 7) all cases handled?
 * 8) default returns/logs/throws error
 * 9) Unspecified/undefined/implementation-defined/non-standard behavior or extensions
 * 10) C++
 * 11) Fourth pass: exception handling:
 * 12) In destructors.
 * 13) Exception catching boundaries?
 * 14) Are resources RAII guarded.
 * 15) Fifth pass: performance.
 * 16) C++11: perfect forwarding & temporaries.
 * 17) How does it scale?
 * 18) Initialized data members.
 * 19) The big 6 (default constructor, destructor, copy/move constructors/operators).
 * 20) Virtual destructor?
 * 21) Resource handling: RAII.
 * 22) Where is created & destructed? Lifetime management?
 * 23) Thorough const & volatile.
 * 24) Are predicates stateless.
 * 25) Use size_t for std::string positions.
 * 26) Why casts?
 * 27) Do not trace under lock
 * 28) Do not destroy objects under lock
 * 29) Platform specific
 * 30) POSIX/SUS
 * 31) Interrupted calls.
 * 32) Spurious wake-ups.
 * 33) Calls known to have race conditions.

Tools

 * Source Code Audit
 * CERT C Secure Coding Standard

Links

 * Undefined Behavior

/proc
A pseudo-filesystem exposing information about running processes:

sysstat, sar

 * Site

addr2line
Given an address in an executable or an offset in a section of a relocatable object, addr2line translates it into file name and line number.

c++filt
A tool to demangle symbol names.

objdump

 * Disassemble binary, with source code: objdump -C -S -r -R -l 

Varia

 * Source Code Audit

Links

 * Bunch of tools
 * dstat

Abbreviations

 * HLE: Hardware Lock Elision
 * HT: Hyper Threading
 * RTM: Restricted Transactional Memory
 * TSX: Intel Transactional Synchronization Extension