This chapter describes heap corruption detection and covers the following topics:
Due to the dynamic nature of allocating and deallocating memory, the heap is vulnerable to these common corruption problems:
To detect heap corruption problems, you need to relink your executable with a special WorkShop malloc library (-lmalloc_cv) instead of the standard malloc library (-lmalloc). By default, the library always catches these errors:
malloc call failing (returning NULL)
realloc call failing (returning NULL)
realloc call with an address outside the range of heap addresses returned by malloc or memalign
memalign call with an improper alignment
free call with an address that is improperly aligned
free call with an address outside the range of heap addresses returned by malloc or memalign
If you additionally set the MALLOC_FASTCHK environment variable, you can detect these errors:
free or realloc calls where the words prior to the user block have been corrupted
free or realloc calls where the words following the user block have been corrupted
free or realloc calls where the address is that of a block that has already been freed. This error may not always be detected if the area around the block is reallocated after it was first freed.
You can compile your executable from scratch as follows:
cc -g -o targetprogram targetprogram.c -lmalloc_cv |
You can also relink it by using:
ld -o targetprogram targetprogram.o -lmalloc_cv ... |
An alternative to rebuilding your executable is to use the environment variable _RLD_LIST to link the -lmalloc_cv library. See the reference (man) page for rld(1).
After compiling, you invoke the Debugger with your executable as the target. In Execution View, you can set environment variables to enable different levels of heap corruption detection from within the malloc library, as follows:
| MALLOC_CLEAR_FREE |
| |
| MALLOC_CLEAR_FREE_PATTERN <pattern> |
| |
| MALLOC_CLEAR_MALLOC |
| |
| MALLOC_CLEAR_MALLOC_PATTERN <pattern> |
| |
| MALLOC_FASTCHK |
| |
| MALLOC_MAXMALLOC n |
| |
| MALLOC_NO_REUSE |
| |
| MALLOC_TRACING |
| |
| MALLOC_VERBOSE |
|
For further information, see the reference page for malloc_cv.
If you are using the -lmalloc_cv library, you can use the Trap Manager to set a stop trap at the exit from the function cvmalloc_error which is called when an error is detected. Errors are detected only during calls to heap management routines, such as malloc() and free(). Some kinds of errors, such as overruns, are not detected until the block is freed or realloced.
When you run the program, it will halt at the stop trap if a heap corruption error is detected. The error and the address are displayed in Execution View. You can also examine the Call Stack View at this point to get stack information. To find the next error, click the Continue button.
If you need more information to isolate the error, set a watchpoint trap to detect a write at the displayed address; then run your program again. Use MALLOC_CLEAR_FREE and MALLOC_CLEAR_MALLOC to catch problems from attempts to access uninitialized or freed memory.
 | Note: You can run programs linked with -lmalloc_cv library outside of the Debugger. The trade-off is that you have to browse through the stderr messages and catch any errors through visual inspection. |
This tutorial demonstrates how to detect corruption errors, using a program called corrupt. The corrupt program has already been linked with the WorkShop malloc library (libmalloc_cv). Its listing follows:
#include <string.h>
void main (int argc, char **argv)
{
char *str;
int **array, *bogus, value;
/* Let us malloc 3 bytes */
str = (char *) malloc(strlen("bad"));
/* The following statement writes 0 to the 4th byte */
strcpy(str, "bad");
free (str);
/* Let us malloc 100 bytes */
str = (char *) malloc(100);
array = (int **) str;
/* Get an uninitialized value */
bogus = array[0];
free (str);
/* The following is a double free */
free (str);
/* The following statement uses the uninitialized value as a pointer */
value = *bogus;
} |
Go to the directory /usr/demos/WorkShop/mallocbug.
Invoke the Debugger by typing:
cvd corrupt &
The Debugger Main View window displays with corrupt as the target executable.
Open the Execution View window (if it is minimized) and set the MALLOC_FASTCHK and MALLOC_CLEAR_MALLOC environment variables.
If you are using the C shell, type:
setenv MALLOC_FASTCHK setenv MALLOC_CLEAR_MALLOC
If you are using the Korn or Bourne shell, type:
MALLOC_FASTCHK=MALLOC_CLEAR_MALLOC=export MALLOC_FASTCHK MALLOC_CLEAR_MALLOC
Select "Trap Manager" from the Views menu in Main View.
Type the following command in the Trap field of the Trap Manager window and click Add:
Stop exit cvmalloc_error
A stop trap is set at the exit from the malloc library routine cvmalloc_error. This stops the process when a heap corruption error is detected. The Trap Manager is shown in Figure 8-1 with the stop trap set.
Click Run in the Main View control panel to start program execution and observe Execution View.
A heap corruption is detected and the process stops at one of the traps. The type of error and its address display in Execution View as shown in Figure 8-2.
Select "Call Stack" from the Views menu in Main View.
Call Stack View is opened displaying the call stack frame at the time of the error (see Figure 8-3).
Click Continue in the Main View control panel and watch Execution View and Call Stack View.
The process continues from the stop at the boundary overrun warning until it hits the next trap where an erroneous free error occurs
Click Continue again and watch Execution View and Call Stack View.
This time the process stops at a bus error. The PC stops at the statement:
value=*bogus
because bogus was set to an uninitialized value.
Type p &bogus at the Debugger command line at the bottom of the Main View window.
This gives us the address for the variable bogus and has been done in Figure 8-4. We need the bad address so that we can set a watchpoint to find out when it is written to. (Note in this example that the address is 0x7fffaef4—your address will be different.)
Deactivate the stop trap by clicking the toggle button next to the trap description in the Trap Manager window, and click Kill in Main View to kill the process.
Type the following command in the Trap field in the Trap Manager using the address you obtain from the Debugger command line (see Figure 8-4) and click Add:
stop watch address 0x7fffaef4 for write
Use the actual address from your system, not the one in the tutorial. This sets a watchpoint that is triggered if a write is attempted at that address.
Click Run and observe Main View.
The process stops at the point where the variable bogus gets a bad value. The details of the error display in the Main View Status field (see Figure 8-5).
