gdb.asc from EmStar at Krugle

Show gdb.asc syntax highlighted

Using GDB with EmStar
=====================
The EmStar Team
$Id: gdb.asc,v 1.2 2005/09/01 01:59:28 girod Exp $

This is not intended to be a detailed guide to using gdb.  However, I wanted to document my own experiences in the hope that it may be useful for others in the future.

EmRun operates by launching a number of different processes, as
opposed to spawning threads.  This means that running `emrun` from
within GDB is not generally what you want to do.  Rather, you must use
GDB to analyse the process that is running the specific EmStar module
you're having trouble with.

Attaching GDB to a Running Process
----------------------------------

GDB does provide a mechanism to allow you to attach it to a running process, like this.

1) After starting EmRun with your desired run file, get the pid of the process you wish to attach to, either by examining the EmRun output, or by doing: 

--------------
cat /dev/emrun/status
--------------

This returns (amongst other information) the pid of all spawned processes.

2) Run gdb

3) At the gdb prompt, type:

--------------
(gdb) file <path to binary of the process you're debugging>
(gdb) attach <pid of process to attach to>
--------------

You can now set breakpoints, and debug as you normally would this spawned process

NOTE: You can debug as many different processes as you like, just start up a new gdb instance for each process you wish to debug.

Debugging Startup Behavior
--------------------------

Attaching as described above works fine if you are interested in
'steady-state' behaviour (in other words, it doesn't matter to you at
which point in program execution you successfully attach to your
running process).  However, if the problem you are debugging 
occurs at the very beginning of your run, the process will have crashed
before you can attach.  

To address this, there is a command line option parsed in the `misc_init()`
call that will stop your program and wait for you to attach with GDB.
Simply add the directive '--stop-on-startup' to the list of command
line options specified using the cmd function in your EmRun macro.
For example, the cmd command may now look like:

-------
cmd = "$debug mote/hostmoted $args -p $port -m $type -r $index --stop-on-startup";
-------

Note that many EmRun macros include an `$args` argument for exactly
this purpose, so this can be done more conveniently (without changing
the macro definition itself) this way:

-------
&mote_hostmote(serialPort=/dev/ttyS0,args="--stop-on-startup");
-------

This stopping behavior is implemented in the `misc_init()` function,
and thus won't stop your program until it enters that function call
(but typically this is called very early on in an EmStar program).

Debugging Post-Mortem
---------------------

Sometimes you don't know which process will crash ahead of time and
it's inconvenient to trace them all.  To handle this case there are
two techniques.  

First, you can analyse a core file that is left behind by a process
that has died.  This is done by starting gdb as you normally would
and then issuing the `target core <corefile>` command:

--------------
(gdb) file <path to binary of the process you're debugging>
(gdb) target core <corefile>
--------------

Typically the corefile will show up in `emstar/obj.<arch>/core`.
If you have more than one there they will be written as `core.PID`
for each separate process.  

You can also "trap" process as they die.  To do this, add the `--trap`
argument to the emrun command line.  When trap mode is enabled,
deadly signals such as SIGSEGV, SIGBUS, and SIGFPE are trapped by 
a signal handler and the process emits an error message saying 
that it has "trapped" and enters a waiting loop.  

You can then attach using GDB at your leisure and use the `kill` shell
command to issue signal SIGUSR1 to the process, then continue in GDB a
few times.  Eventually you will reenter the crash.  This technique is
also described in the 
link:/emstar/tut/debug/index.html[tutorial on EmStar Debugging].


What not to do with GDB
-----------------------

I can also provide some tips on things not to do:

1) Be very careful when placing breakpoints in emrun/emrun/emrun_proc.c where processes are being spawned.  There is some signal complexity in here which plays havoc with gdb, and may cause your program to crash.

2) Make sure you do not attach gdb to a process before the process context has been replaced by the call to execvp(...).  Unsurprisingly, gdb does not handle this case, and it will crash itself and your program.


Another idea that doesn't quite work yet
----------------------------------------

A suggestion was made that code could be added to a child process so that the child would spawn another child process to run gdb, and attach gdb to itself.  Below is some sample code to do that, although I was not able to figure out how to spawn gdb in a new terminal (I was also unable to create new terminals in this way - the terminal terminated itself as soon as it was spawned).

So in other words, this does work, but gdb runs in the original 'parent' terminal, which will no doubt cause problems if your parent process generates output text.

------------------

#include <sys/types.h>
#include <unistd.h>

int main()
{

  int b;
  pid_t pid = getpid();
  char *arg_list[4];
  char *arg_list1[4];
  char str_to_attach1[1000];  
  char str_to_attach2[1000];  
  char file_name[1000] = "gdb";
  
  arg_list[0] = &file_name[0];
  arg_list[1] = &str_to_attach1[0];
  arg_list[2] = &str_to_attach2[0];
  arg_list[3] = NULL;
  
  printf("I am process b, pid = %d\n", pid);
  
  sprintf(str_to_attach1, "attach");
  sprintf(str_to_attach2, "%d", pid);

  pid = fork();
  if (pid < 0)
    {
      perror("Unable to fork");
      return -1;
    }

  if (pid == 0)
    {
      // this is the child - start gdb and attach to self
      execvp("gdb", arg_list);
    }
  else
    {
      // this is the parent
      while (1)
	{
	  b = b+1;
	  if (b > 10000)
	    {
	      b = 0;
	    }
	  sleep(2);
	}      
    }

  return 0;
}


------------------
See more files for this project here