A module is something loaded into memory. It can be a library, binary, font,
etc. We need to get the debug information for a given module in order to know
what symbols correspond to which addresses.
For Breakpad SYM files, modules are denoted by a debug filename and debug id.
Each debug filename and debug id pair corresponds to a path in the symbols
store. The full URL comes from taking the base url, appending the debug
filename, appending the debug id, and then appending the debug filename again
with a .sym extension.
Given a set of modules and a set of stacks, symbolication is the act of
fetching the debug symbols files for the modules and then looking up the
symbols corresponding to the module offsets.
If you’re trying to symbolication large numbers of things and it’s going slowly,
here are some things you can try:
Always set the User-Agent.
Always set the user agent. If possible, include the url to the git repository
where your code is stored.
Wait and retry on HTTP 429 and HTTP 5xx responses.
If you get an HTTP 429 or HTTP 5xx response, wait and retry again. You could
use a backoff of (0s, 1s, 2s, 3s, 4s, 5s).
You should always get back a JSON response. If you don’t, treat that like
a temporary failure, wait a bit and try again.
If you’re getting a 200 response, but some frames aren’t symbolicated, then
either there are no debugging symbols available for that module or the
debugging symbols for that module are malformed.
You can see if we have debugging symbols and check to see if they’re
well-formed by requesting the module from Mozilla Symbols Server.
Batch symbolication requests as a single request with multiple jobs.
If you batch your symbolication requests it’ll reduce the HTTP
request/response overhead and improve the likelihood that the request
handler can take advantage of the symcache LRU cache.
Further, it’s best to batch requests for the same platform/buildid/product.
That’ll reduce the number of times the request handler has to parse the xul
module sym file to at most 1.
For example, if you were trying to symbolicate 1,000,000 crash pings from
yesterday, you should:
sort them by platform/buildid/product
for each platform/buildid/product tuple:
for each N crash pings
create a symbolication request
perform the symbolication request
Currently, a good N is 1,000, but it depends on the size of the stacks and
what other modules are involved. You may find a lower N is better.
Debug – if you add Debug:true to the headers, then symbolication
will also return debug information about cache lookups, how many downloads,
timings, and some other things
HTTP/1.1200OKContent-Type:application/json{"symbolicatedStacks":[["XREMain::XRE_mainRun() (in xul.pdb)","KiUserCallbackDispatcher (in wntdll.pdb)"]],"knownModules":[true,true]}