Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement the solidity compilation cache #6129

Open
wants to merge 56 commits into
base: v-next
Choose a base branch
from

Conversation

galargh
Copy link
Member

@galargh galargh commented Jan 9, 2025

  • Because this PR includes a bug fix, relevant tests have been included.
  • Because this PR includes a new feature, the change was previously discussed on an Issue or with someone from the team.
  • I didn't do anything of this.

In this PR, I implement the solidity compilation cache as outlined in the associated design doc.

The implementation diverges from the design doc in 1 areas:

Copy link

changeset-bot bot commented Jan 9, 2025

⚠️ No Changeset found

Latest commit: cbe10ee

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

💥 An error occurred when fetching the changed packages and changesets in this PR
Some errors occurred when validating the changesets config:
The package "@nomicfoundation/ignition-core" depends on the ignored package "hardhat", but "@nomicfoundation/ignition-core" is not being ignored. Please add "@nomicfoundation/ignition-core" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-ethers" depends on the ignored package "hardhat", but "@nomicfoundation/hardhat-ignition-ethers" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-ethers" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-viem" depends on the ignored package "hardhat", but "@nomicfoundation/hardhat-ignition-viem" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-viem" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition" depends on the ignored package "hardhat", but "@nomicfoundation/hardhat-ignition" is not being ignored. Please add "@nomicfoundation/hardhat-ignition" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-ethers" depends on the ignored package "@nomicfoundation/hardhat-ethers", but "@nomicfoundation/hardhat-ignition-ethers" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-ethers" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition" depends on the ignored package "@nomicfoundation/hardhat-network-helpers", but "@nomicfoundation/hardhat-ignition" is not being ignored. Please add "@nomicfoundation/hardhat-ignition" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition" depends on the ignored package "@nomicfoundation/hardhat-verify", but "@nomicfoundation/hardhat-ignition" is not being ignored. Please add "@nomicfoundation/hardhat-ignition" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-viem" depends on the ignored package "@nomicfoundation/hardhat-viem", but "@nomicfoundation/hardhat-ignition-viem" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-viem" to the `ignore` option.
The package "@nomicfoundation/ignition-core" depends on the ignored package "@nomicfoundation/eslint-plugin-hardhat-internal-rules", but "@nomicfoundation/ignition-core" is not being ignored. Please add "@nomicfoundation/ignition-core" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-ethers" depends on the ignored package "@nomicfoundation/eslint-plugin-hardhat-internal-rules", but "@nomicfoundation/hardhat-ignition-ethers" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-ethers" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition-viem" depends on the ignored package "@nomicfoundation/eslint-plugin-hardhat-internal-rules", but "@nomicfoundation/hardhat-ignition-viem" is not being ignored. Please add "@nomicfoundation/hardhat-ignition-viem" to the `ignore` option.
The package "@nomicfoundation/hardhat-ignition" depends on the ignored package "@nomicfoundation/eslint-plugin-hardhat-internal-rules", but "@nomicfoundation/hardhat-ignition" is not being ignored. Please add "@nomicfoundation/hardhat-ignition" to the `ignore` option.

Copy link

vercel bot commented Jan 9, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hardhat ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 6, 2025 2:39pm

Copy link
Contributor

github-actions bot commented Jan 10, 2025

hardhat

Total size of the bundle: 213M
Total number of dependencies (including transitive): 53

List of dependencies (sorted by size)
208M	total
29M	@ignored/edr-optimism-linux-x64-musl
29M	@ignored/edr-optimism-linux-x64-gnu
26M	@ignored/edr-optimism-linux-arm64-musl
26M	@ignored/edr-optimism-linux-arm64-gnu
22M	@ignored/edr-optimism-win32-x64-msvc
20M	esbuild
20M	@ignored/edr-optimism-darwin-x64
19M	@ignored/edr-optimism-darwin-arm64
2.8M	@sentry/tracing
2.5M	micro-eth-signer
1.9M	@noble/curves
1.7M	undici
1.2M	@sentry/types
1.2M	@noble/hashes
932K	@sentry/node
920K	@sentry/utils
856K	zod
856K	@ignored/hardhat-vnext-utils
624K	micro-packed
576K	tsx
548K	@sentry/core
544K	fast-equals
492K	@scure/bip39
460K	@ignored/edr
368K	ethereum-cryptography
344K	@sentry/hub
324K	@ignored/hardhat-vnext-errors
320K	enquirer
284K	semver
264K	@ignored/edr-optimism
192K	ws
180K	adm-zip
168K	@scure/base
136K	get-tsconfig
96K	@scure/bip32
92K	chalk
88K	tslib
88K	@sentry/minimal
76K	agent-base
72K	@nomicfoundation/solidity-analyzer
68K	debug
64K	lru_map
64K	https-proxy-agent
60K	@ignored/hardhat-vnext-zod-utils
56K	rfdc
48K	ansi-colors
40K	resolve-pkg-maps
36K	p-map
32K	cookie
24K	strip-ansi
24K	env-paths
24K	ansi-regex
20K	ms


import { ResolvedFileType } from "../../../../types/solidity.js";

export class ProjectResolvedFileImplementation implements ProjectResolvedFile {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context

I add implementations of ProjectResolvedFile and NpmPackageResolvedFile interfaces here. These are necessary in order to lazily store content hashes on them, as proposed as part of the getBuildId optimisations.

@@ -35,14 +35,15 @@ export interface ResolvedNpmPackage {
*/
export enum ResolvedFileType {
PROJECT_FILE = "PROJECT_FILE",
NPM_PACKGE_FILE = "NPM_PACKAGE_FILE",
NPM_PACKAGE_FILE = "NPM_PACKAGE_FILE",
}

/**
* A file that's part of the Hardhat project (i.e. not installed through npm).
*/
export interface ProjectResolvedFile {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context

I add getContentHash() methods on the ProjectResolvedFile and NpmPackageResolvedFile interfaces here. These functions are needed to implement the getBuildId optimisations.

@@ -17,17 +15,22 @@ export async function keccak256(bytes: Uint8Array): Promise<Uint8Array> {
*
* This function is primarily intended for generating unique identifiers from
* a given input string.
* It uses the MD5 hash algorithm, which is not cryptographically secure, but
* It uses the SHA-1 hash algorithm, which is not cryptographically secure, but
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this LGTM

buildId,
contractArtifactsGenerated:
contractArtifactsGenerated.get(publicSourceName) ?? [],
warnings: errors,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self-answer: yes, these warnings have been remapped

Comment on lines +722 to +724
* If we notice that the amount of write failures is too high, or that the
* cache is not getting cleaned up often enough, we should reconsider this
* approach.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, I included some of the reasons why we might have to end up awaiting the cache update after all. I think we can start with this version and reconsider if/when needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:ready This issue is ready to be worked on v-next A Hardhat v3 development task
Projects
Status: Backlog
Development

Successfully merging this pull request may close these issues.

Compilation caching in the local file system
2 participants