CompilerGPT

Compiler optimization reports are an important tool for performance engineers. However, they are often too complex for non compiler experts. CompilerGPT attempts to harness the power of LLM AI models to automatically attempt optimizations suggested by the optimization reports.

CompilerGPT is a framework that submits compiler optimization reports (i.e., Clang) and the source code to an LLM. The LLM is prompted to prioritize the findings in the optimization reports and then to make changes in the code accordingly. An automated test harness validates the changes. The test harness provides feedback to the LLM on any errors that were introduced to the code base.

CompilerGPT iterates with the LLM a given number of times and reports on the obtained correctness and perfomance results of each iteration.

CompilerGPT calls AI modules through scripts which makes it flexible to use with different frameworks. Most connections are based on CURL. A legacy configuration for OpenAI's Python framework also exists. CompilerGPT's auto-configuration currently supports OpenAI, Anthropic, openrouter, and ollama.

Getting started

Requirements:

• C++ compiler supporting C++20, e.g. GCC 12
  • boost json (boost > 1.78)
• curl

Directory structure:

• scripts: scripts to interact with AI (e.g., gpt and claude)
• src: C++ sources (compgpt and tools)
• tests: test codes to illustrate the working of CompilerGPT

GPT Environment: Default queries will use OpenAI's API (in scripts/gpt4/). Three environemnt variables control the invocation:

• OPENAI_API_KEY the openai key for queries
• SSL_CERT_FILE (optional) the location of a root certificate to avoid SSL connection errors.
  typical error message on Linux: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain

Claude Environment:

• ANTHROPIC_API_KEY the key to support Claude requests

Openrouter.ai Environment:

• OPENROUTER_API_KEY the key to support openrouter.ai requests

Build CompilerGPT:

export BOOST_HOME="/path/to/boost/install/dir"
make

Call CompilerGPT:

compgpt.bin --config=configfile.json c++-source-file

The configuration is read from the json file. The config file specifies how to interact with a target compiler and the LLM system.

A default configuration can be created by

compgpt.bin --create-config --config=configfile.json

Claude is an alternative to the GPT model. It can be used in a similar way than gpt, but it requires a different configuration. A default configuration file can be created by

compgpt.bin --create-config --config=claude.json --config:ai=claude --config:model=claude-3-5-sonnet-20241022

CompilerGPT works with clang and gcc. The configuration can be generated by providing the --config:compiler setting.

compgpt.bin --create-config --config=gcc-claude.json --config:ai=claude --config:compiler=/usr/bin/g++

New configurations can be derived from existing ones by using --config:from=configfile.json . This can be useful to create various configurations for different AI components or compilers that use the same prompts and evaluation scripts.

compgpt.bin --create-config --config=ollama.json --config:ai=ollana --config:model=llama3.3 --config:from=claude.json

Other use cases

In addition, compgpt can be setup to use AI for translation projects. Currently, the config file needs to be tweaked manually to support that use case.

License

CompilerGPT is distributed under a BSD 3 license.

SPDX-License-Identifier: BSD-3-Clause

See the LICENSE file for details.

Third-party benchmarks distributed with CompilerGPT retain the original licensing. See the respective directories for license details.

Project status

1.x Demonstrates the basic workflow

Source Code Release

LLNL-CODE-2000601