TL;DR Use convolution to find type candidates, then solve system of equations to refine the result.
You are given a set of memory regions. And set of types (structures) that can possibly be present in the regions.
You can find locations of possible pointers (x86_64):
- the positions are aligned to 8B,
- values are from
- values are of interest are aligned too.
You can thus distinguish pointer and non-pointer locations -- in the memory thanks to criteria above, within a structure type thanks to its declaration.
c(x, t) a score that at memory location x is stored object of type t -- basically you NXOR pointer pattern of the type and
[x, sizeof(t)) and count all ones.
This will give you first estimate of what object might be in memory.
Further we define improved score as
s(x, t) = c(x, t) + sum(s(*(x+t), type) for offset, type in t.pointers)
I am not sure about analytic solution of
s(x, t), however, an iterative method might converge to a valid result.
Why all this? Consider you have a core dump, where you mark all reachable (including glibc malloc internal structures) memory similarly to garbage collection operation. Then you take allocated (read allocator data) but unmarked memory and process it as desribed above. Voila, you should know now what types of objects your program leaked.
Looking for mad skills in:
Nothing? Add some keywords!
This project is part of:
Hack Week 15
This project is one of its kind!