Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Llvm16 pr #23

Open
wants to merge 10 commits into
base: www
Choose a base branch
from
Open

Llvm16 pr #23

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
eaf9ec6
Update UsingEnzyme.md
davidedelvento Feb 3, 2024
f44dd62
Multisource example
davidedelvento Feb 3, 2024
97860c1
Making Enzyme version and location generic
davidedelvento Feb 3, 2024
fe0dedf
removing double-spaces inadvertently included earlier
davidedelvento Feb 3, 2024
66b1d34
Added a section title to make it easy to find an important part
davidedelvento Feb 3, 2024
2880494
The section of the previous commit was at the wrong paragraph... fixing
davidedelvento Feb 3, 2024
a2c5afa
Stubs section for advanced options
davidedelvento Feb 3, 2024
84f4afc
Updated all the calls to LLVM16
davidedelvento Feb 3, 2024
dfea349
more professional wording
davidedelvento Feb 3, 2024
76b81c9
Minor word changes, typo
davidedelvento Feb 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 111 additions & 16 deletions content/getting_started/UsingEnzyme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,17 @@ entry:
...
```

## Performing AD Enzyme
## Performing autodifferentiation (AD) with Enzyme using LLVM natively
We can now run Enzyme to differentiate our LLVM IR. The following command will load Enzyme and run the differentiation transformation pass. Note that `opt` should be the path to whatever opt was creating by the LLVM you built Enzyme against. If you see a segfault when trying to run opt, this is likely an issue in LLVM's plugin infrasture. Please see [the installation guide](/getting_started/Installation) for more information on how to resolve this.

At version v16, LLVM changed the CLI syntax for invoking the optimization passes. Hence the following boxes contain two options. For LLVM versions before v16 run:
```sh
opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -o output.ll -S
```
For LLVM versions at v16 and subsequent instead run:
```sh
opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -o output.ll -S
```

Taking a look at `output.ll`, we find the following:

Expand Down Expand Up @@ -133,17 +138,34 @@ We can then compile this to a final binary as follows:
clang output_opt.ll -o a.exe
```

For ease, we could combine the final optimization and bianry execution into one command as follows.
For ease, we could combine the final optimization and binary execution into one command as follows.
```sh
clang output.ll -O3 -o a.exe
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call this C++ native?

## Performing AD with Enzyme (the easy, one-step way)
Moreover, using Enzyme's clang plugin, we could automate the entire AD and compilation in a single command. Using the clang plugin should be done by default as it improves the user experience as well as various default performance options. However, the example above is still useful to understand how Enzyme works on LLVM.

```sh
clang test.c -fplugin=/path/to/Enzyme/enzyme/build/Enzyme/ClangEnzyme-<VERSION>.so -o a.exe
```

When compiling multiple files together, as it is typically done in larger projects, e.g. with a Makefile, one would do something like the following.

For LLVM versions before v16:
```sh
clang -c -flto file1.c -O2 -o file1.o
clang -c -flto file2.c -O2 -o file2.o
clang -fuse-ld=lld -flto file1.o file2.o -O2 -o a.exe -Wl,-mllvm=-load=/path/to/Enzyme/enzyme/build/Enzyme/LLDEnzyme-<VERSION>.so
```

For LLVM versions at v16 and newer:
```sh
clang -c -flto file1.c -O2 -o file1.o
clang -c -flto file2.c -O2 -o file2.o
clang -fuse-ld=lld -flto file1.o file2.o -O2 -o a.exe -Wl,--load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLDEnzyme-<VERSION>.so
```

Note that if using the LLVM plugin, each version of LLVM will have slightly different command line flags to specifying plugins. See [the FAQ](/getting_started/Faq) for more information. If using the clang plugin, the same command should work independently of version.

## Advanced options
Expand All @@ -156,67 +178,110 @@ Enzyme has several advanced options that may be of interest.

The `enzyme-preopt` option disables the preprocessing optimizations run by the Enzyme pass, except for the absolute minimum neccessary.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-preopt=1
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-preopt=0
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-preopt=1
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-preopt=0
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-preopt=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-preopt=0
```

#### Forced Inlining

The `enzyme-inline` option forcibly inlines all subfunction calls. The `enzyme-inline-count` option limits the number of calls inlined by this utility.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-inline=1
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-inline=1 -enzyme-inline-count=100
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-inline=1
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-inline=1 -enzyme-inline-count=100
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-inline=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-inline=1 -enzyme-inline-count=100
```

#### Compressed Bool Cache

The `enzyme-smallbool` option allows Enzyme's cache to store 8 boolean (i1) values inside a single byte rather than one value per byte.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-smallbool=1
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-smallbool=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-smallbool=1
```


### Semantic options

#### Loose type analysis

The `enzyme-loose-types` option tells Enzyme to make an educated guess about the type of a value it cannot prove, rather than emit a compile-time error and fail. This can be helpful for starting to bootstrap code with Enzyme but shouldn't be used in production as Enzyme may make an incorrect guess and create an incorrect gradient.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-loose-types=1
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-loose-types=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-loose-types=1
```


#### Assume inactivity of undefined functions

The `enzyme-emptyfn-inactive` option tells activity analysis to assume that all calls to functions whose definitions aren't available and aren't explicitly given a custom gradient via metadata are assumed to be inactive. This can be useful for assuming printing functions don't impact derivative computations and provide a performance benefit, as well as getting around a compile-time error where the derivative of a foreign function is not known. However, this option should be used carefully as it may result in incorrect behavior if it is used to incorrectly assume a call to a foreign function doesn't impact the derivative computation. As a result, the recommended way to remedy this is to mark the function as inactive explicitly, or provide a custom gradient via metadata.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-emptyfn-inactive=1
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-emptyfn-inactive=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-emptyfn-inactive=1
```


#### Assume inactivity of unmarked globals

The `enzyme-globals-default-inactive` option tells activity analysis to assume that global variables without an explicitly defined shadow global are assumed to be inactive. Like `enzyme_emptyfnconst`, this option should be used carefully as it may result in incorrect behavior if it is used to incorrectly assume that a global variable doesn't contain data used in a derivative computation.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-globals-default-inactive=1
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-globals-default-inactive=1
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-globals-default-inactive=1
```

#### Cache behavior

#### Cache behavior (this does not appear to be available in more recent versions of Enzyme)

The `enzyme-cache-never` option tells the cache to recompute all load values, even if alias analysis isn't able to prove the legality of such a recomputation. This may improve performance but is likely to result in incorrect derivatives being produced as this is not generally true.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-cache-never=1
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-cache-never=1
```

In contrast, the `enzyme-cache-always` option tells the cache to still cache values that alias analysis and differential use analysis say are not needed to be cached (perhaps being legal to recompute instead). This will usually decrease performance and is intended for developers in order to catch caching bugs.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-cache-always=1
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-cache-always=1
```

### Debugging options for developers
Expand All @@ -225,8 +290,18 @@ $ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-cache-always=1

This option prints out functions being differentiated before preprocessing optimizations, after preprocessing optimizations, and after being synthesized by Enzyme. It is mostly use to debug the AD process.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-print
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-print
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-print
```

For either version you will see and output like the following:
```
prefn:

; Function Attrs: norecurse nounwind readnone uwtable
Expand All @@ -241,8 +316,18 @@ entry:

This option prints out the results of activity analysis as they are being derived. The output is somewaht specific to the analysis pass and is only intended for developers.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-print-activity
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-print-activity
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-print-activity
```

For either version you will see and output like the following:
```
in new function diffesquare nonconstant arg double %0
VALUE nonconst from arg nonconst double %x
checking if is constant[3] %mul = fmul double %x, %x
Expand All @@ -258,8 +343,18 @@ couldnt decide nonconstants(3): %mul = fmul double %x, %x

This option prints out the results of type analysis as they are being derived. The output is somewaht specific to the analysis pass and is only intended for developers.

For LLVM versions before v16 run:
```sh
$ opt input.ll -load=./Enzyme/LLVMEnzyme-7.so -enzyme -enzyme-print-type
$ opt input.ll -load=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so -enzyme -enzyme-print-type
```

For LLVM versions at v16 and newer:
```sh
$ opt input.ll --load-pass-plugin=/path/to/Enzyme/enzyme/build/Enzyme/LLVMEnzyme-<VERSION>.so --passes=enzyme -enzyme-print-type
```

For either version you will see and output like the following:
```
analyzing function square
+ knowndata: double %x : {[-1]:Float@double} - {}
+ retdata: {}
Expand Down