Re: Kernel extension debugging tool?
Re: Kernel extension debugging tool?
- Subject: Re: Kernel extension debugging tool?
- From: Sam Vaughan <email@hidden>
- Date: Tue, 15 Nov 2005 11:29:29 +1100
On 14/11/2005, at 7:03 AM, Daniel Azuelos <email@hidden> wrote:
What's the best way, method to debug a kernel extension?
I've a slow memory leak to locate precisely and l can't wait the many
hours it needs to lead to a real system problem.
For memory leaks I'd recommend adding some memory debugging code to
your kernel extension. How far you take it depends on how complex
your kernel extension is and therefore whether you think you'll get
enough return on the investment. I recently had a slow mbuf leak
that prompted me to beef up our memory debugging code, and it's now
at a point where I can monitor outstanding allocations on a live
system grouped by the symbolic backtraces of all the allocation paths
in the code. This makes memory leaks stand out like a sore thumb!
Firstly, you'll need to funnel all your memory allocations and frees
through the same functions so you've only got one place to add
debugging code that will catch everything. I changed our code to use
kernel memory allocation for mbufs to make sure I could catch mbuf
misuse as well.
Next, change the code to allocate more space than the original
request, allowing you to store extra information like the current
backtrace and the size of the allocation. At this point you might
want to add space to both ends and poison it with magic values like
0xdefec8ed or something similarly childish. ;o) Poisoning memory
after allocating it and after freeing it also helps trap code that
reuses freed memory. It'd be even nicer to have read-only guard
pages to generate VM exceptions on writes outside the bounds of
allocated memory, but putting an entire page on either side of small
allocations is going to fill up some of the kernel's zones very quickly.
Add each new allocation's address to a hash table, and remove it
again when it is freed. Verify that your free calls specify the same
size as the original allocation. You can use this hash table at
unload/quiesce time to check for missing frees.
Maintain a list of unique allocation backtraces. I just store four
addresses for each backtrace since going further back than that often
takes you outside your kext and into the kernel, giving you unique
backtraces where you would consider them to be the same. Alongside
the unique backtrace entries, you can maintain allocation and free
counts, total amount allocated, etc.
Add a sysctl to dump out this information. Write a script to parse
it, calling `nm /S/L/E/your.kext | grep -wi t | sort` to get your
symbols, and `kextstat | grep your.bundle.id | awk '{print \$3}'` to
get your load address. Subtract your load address plus one page
(0x1000) from all the backtrace addresses your kext dumped out, and
you can then look them up in the symbol list you got from nm. Print
it all out sorted by e.g. outstanding allocations, total memory
allocated, total allocation count etc. and you can get a very handy
representation of your kext's memory usage as it runs.
Of course you'll want to turn all this stuff off in your release
builds because it slows things down a bit. Having said that, it had
far less of an impact than I anticipated.
Good luck!
Sam
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Macnetworkprog mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden