Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Golang: a bit of house keeping #38766

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Conversation

hypnoce
Copy link
Contributor

@hypnoce hypnoce commented Mar 16, 2025

Commit Message: Align all loggers to use golang component Id. Improve sendLocalReply by using go memory pinning instead of string copy.
Additional Description:
Risk Level: Low
Testing: no need as it is only refactoring
Docs Changes:
Release Notes: updated current.yaml
Platform Specific Features: N/A

@hypnoce
Copy link
Contributor Author

hypnoce commented Mar 16, 2025

@doujiang24 during my coding, I came accros various little cleanup. Let me know if you find them usefull.

Copy link
Member

@doujiang24 doujiang24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hypnoce Cool, thanks for your work.

@@ -203,7 +205,6 @@ func (c *httpCApiImpl) HttpCopyHeaders(s unsafe.Pointer, num uint64, bytes uint6
m[key] = append(v, value)
}
}
runtime.KeepAlive(buf)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove this?

Copy link
Contributor Author

@hypnoce hypnoce Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the buffer is (silently) pinned (no GC and no memory move) until the C call returns and we are garantee that when the C call returns, the golang strings point to the buf underlying byte array, preventing any GC of that array.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, I do not think so, go compiler just make sure it's escaped to heap.

Copy link
Contributor Author

@hypnoce hypnoce Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buf backing array is heap allocated and tracked by go GC (it's considered as a go pointer) since its size is dynamic and cannot be known at compile time.
The buf struct itself is stack allocated but the backing array will not be GCed as it is referenced by the strings.

For instance

func main() {
	str := foo(10)
	runtime.GC()
	runtime.GC()
	println(str)
	runtime.GC()
	runtime.GC()
	str2 := foo2(10)
	runtime.GC()
	runtime.GC()
	println(str2)
	runtime.GC()
	runtime.GC()
}

func foo(bytes int) string {
	buf := make([]byte, bytes)
	buf[0] = 'a'
	buf[1] = 'b'
	data := unsafe.SliceData(buf)
	runtime.SetFinalizer(data, func(*uint8) {
		println("foo1 backing array finalizer called")
	})
	myString := unsafe.String(data, 2)
	return myString
}

func foo2(bytes int) string {
	buf := make([]byte, bytes)
	buf[0] = 'a'
	buf[1] = 'b'
	runtime.SetFinalizer(unsafe.SliceData(buf), func(*uint8) {
		println("foo2 backing array finalizer called")
	})
	myString := "cd"
	return myString
}

This outputs

ab
foo1 backing array finalizer called
foo2 backing array finalizer called
cd

Maybe I missed something but in this exemple buf does not escape to heap while the backing array is kept because referenced by the string.
WDYT ?

@@ -294,7 +295,6 @@ func (c *httpCApiImpl) HttpCopyTrailers(s unsafe.Pointer, num uint64, bytes uint
m[key] = append(v, value)
}
}
runtime.KeepAlive(buf)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

hypnoce added 2 commits March 24, 2025 12:53
…ining instead of string copy.

Signed-off-by: Francois JACQUES <fjacques@murex.com>
Signed-off-by: Francois JACQUES <fjacques@murex.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants