The compiler now uses DW_OP_plus_uconst in the address attribute of
DWARF variables. To be able to understand these addresses, we need to be
able to parse this opcode.
This commit should also improve the reliability of address parsing a
bit.
Previously the code size of the compiler-rt library might be displayed
like this:
```
1050 0 0 0 | 1050 0 | /home/ayke/src/tinygo/tinygo/llvm-project/compiler-rt/lib/builtins
146 0 0 0 | 146 0 | /home/ayke/src/tinygo/tinygo/llvm-project/compiler-rt/lib/builtins/arm
```
With this change, it is displayed nicely like this:
```
1196 0 0 0 | 1196 0 | C compiler-rt
```
This is a big commit that changes the way runtime type information is stored in
the binary. Instead of compressing it and storing it in a number of sidetables,
it is stored similar to how the Go compiler toolchain stores it (but still more
compactly).
This has a number of advantages:
* It is much easier to add new features to reflect support. They can simply
be added to these structs without requiring massive changes (especially in
the reflect lowering pass).
* It removes the reflect lowering pass, which was a large amount of hard to
understand and debug code.
* The reflect lowering pass also required merging all LLVM IR into one
module, which is terrible for performance especially when compiling large
amounts of code. See issue 2870 for details.
* It is (probably!) easier to reason about for the compiler.
The downside is that it increases code size a bit, especially when reflect is
involved. I hope to fix some of that in later patches.
The MachO file format is a bit weird and doesn't store the DWARF debug
information directly in the file. Instead, it has to be looked up in the
original object file. This makes reading the DWARF debug information for
code size usage a bit more difficult. However, it works with this
change.
This is just basic support. It doesn't add support for reading DWARF,
because that's a bit complicated on MacOS (it isn't stored in the file
itself but separately in the object files). But at least this change
makes it possible to easily print executable sizes by section type like
for other operating systems.
> There are two hard things in computer science: cache invalidation,
> naming things, and off-by-one errors.
Because of this bug, sometimes the last object in a section might not be
attributed correctly to a source location.
This fixes a small mistake when calculating binary size for an Xtensa
file. Previously it would exit with the following error:
$ tinygo build -o test.elf -size=short -target=esp32-mini32 examples/serial
panic: runtime error: index out of range [65521] with length 18
Now it runs as expected:
$ tinygo build -o test.elf -size=short -target=esp32-mini32 examples/serial
code data bss | flash ram
2897 0 4136 | 2897 4136
This is fake debug info. It doesn't point to a source location because
there is no source location. However, it helps to correctly attribute
code size usage to particular packages.
I've also updated builder/sizes.go with some debugging helpers.
This commit improves accuracy of the -size=full flag in a big way.
Instead of relying on symbol names to figure out by which package
symbols belong, it will instead mostly use DWARF debug information
(specifically, debug line tables and debug information for global
variables) relying on symbols only for some specific things. This is
much more accurate: it also accounts for inlined functions.
For example, here is how it looked previously when compiling a personal
project:
code rodata data bss | flash ram | package
1902 333 0 0 | 2235 0 | (bootstrap)
46 256 0 0 | 302 0 | github
0 454 0 0 | 454 0 | handleHardFault$string
154 24 4 4 | 182 8 | internal/task
2498 83 5 2054 | 2586 2059 | machine
0 16 24 130 | 40 154 | machine$alloc
1664 32 12 8 | 1708 20 | main
0 0 0 200 | 0 200 | main$alloc
2476 79 0 36 | 2555 36 | runtime
576 0 0 0 | 576 0 | tinygo
9316 1277 45 2432 | 10638 2477 | (sum)
11208 - 48 6548 | 11256 6596 | (all)
And here is how it looks now:
code rodata data bss | flash ram | package
------------------------------- | --------------- | -------
1509 0 12 23 | 1521 35 | (unknown)
660 0 0 0 | 660 0 | C compiler-rt
58 0 0 0 | 58 0 | C picolibc
0 0 0 4096 | 0 4096 | C stack
174 0 0 0 | 174 0 | device/arm
6 0 0 0 | 6 0 | device/sam
598 256 0 0 | 854 0 | github.com/aykevl/ledsgo
320 24 0 4 | 344 4 | internal/task
1414 99 24 2181 | 1537 2205 | machine
726 352 12 208 | 1090 220 | main
3002 542 0 36 | 3544 36 | runtime
848 0 0 0 | 848 0 | runtime/volatile
70 0 0 0 | 70 0 | time
550 0 0 0 | 550 0 | tinygo.org/x/drivers/ws2812
------------------------------- | --------------- | -------
9935 1273 48 6548 | 11256 6596 | total
There are some notable differences:
* Odd packages like main$alloc and handleHardFault$string are gone,
instead their code is put in the correct package.
* C libraries and the stack are now included in the list, they were
previously part of the (bootstrap) pseudo-package.
* Unknown bytes are slightly reduced. It should be possible to reduce
it significantly more in the future: most of it is now caused by
interface invoke wrappers.
* Inlined functions are now correctly attributed. For example, the
runtime/volatile package is normally entirely inlined.
* There is no difference between (sum) and (all) anymore. A better
code size algorithm now counts the code/data sizes correctly.
* And last (but not least) there is a stylistic change: the table now
looks more like a table. Especially the summary should be clearer
now.
Future goals:
* Improve debug information so that the (unknown) pseudo-package is
reduced in size or even eliminated altogether.
* Add support for other file formats, most importantly WebAssembly.
* Perhaps provide a way to expand this report per file, or in a
machine-readable format like JSON or CSV.
See comment in the commit for details. It works around a bug that's been
reported here: https://bugs.llvm.org/show_bug.cgi?id=45336
This is a separate commit so it can easily be reverted if/when this
patch is backported to the LLVM 10 stable branch.
This is a large commit that moves all code directly related to
compiling/linking into a new builder package. This has a number of
advantages:
* It cleanly separates the API between the command line and the full
compilation (with a very small API surface).
* When the compiler finally compiles one package at a time (instead of
everything at once as it does now), something will have to invoke it
once per package. This builder package will be the natural place to
do that, and also be the place where the whole process can be
parallelized.
* It allows the TinyGo compiler to be used as a package. A client can
simply import the builder package and compile code using it.
As part of this refactor, the following additional things changed:
* Exported symbols have been made unexported when they weren't needed.
* The compilation target has been moved into the compileopts.Options
struct. This is done because the target really is just another
compiler option, and the API is simplified by moving it in there.
* The moveFile function has been duplicated. It does not really belong
in the builder API but is used both by the builder and the command
line. Moving it into a separate package didn't seem useful either
for what is essentially an utility function.
* Some doc strings have been improved.
Some future changes/refactors I'd like to make after this commit:
* Clean up the API between the builder and the compiler package.
* Perhaps move the test files (in testdata/) into the builder package.
* Perhaps move the loader package into the builder package.
