Does WithProgress and WithProfile work with HTTP Protocol?

[stephenmcrowe]

Summary

Similar to issue 639, I would like to get the statistics of the number of rows and bytes read per query and log them. My understanding is that information is only available via Profile Info.

Aside: Yes, I understand I can also directly query the ClickHouse query log, but it'd be nice to have it together with the query that is issued.

Observed

The WithProgress and WithProfile callbacks work when connecting over native TCP, but not when using the HTTP protocol.

Code example

package main

import (
	"context"
	"crypto/tls"
	"database/sql"
	"fmt"
	"time"

	clickhousego "github.com/ClickHouse/clickhouse-go/v2"
)

func main() {
        // When using the http port below, the Profile Info is not reported
	conn, err := Connect("db_name:tcp_port", "redacted", "redacted", "default", false)
	err = testSetSetting(conn)
	if err != nil {
		fmt.Printf("testSetSetting err: %v\n", err)
	}
}

func Connect(addr string, user string, pwd string, database string, useTls bool) (*sql.DB, error) {
	var tlsConfig *tls.Config
	if useTls {
		tlsConfig = &tls.Config{}
	}
	// https://clickhouse.com/docs/en/integrations/go#connection-settings
	conn := clickhousego.OpenDB(&clickhousego.Options{
		Addr: []string{addr},
		Auth: clickhousego.Auth{
			Database: database,
			Username: user,
			Password: pwd,
		},
		// Protocol: clickhousego.HTTP,  // uncomment this and the Profile Info is not reported
		TLS: tlsConfig,
		Settings: clickhousego.Settings{
			"max_execution_time": 60,
		},
		DialTimeout: 30 * time.Second,
		Compression: &clickhousego.Compression{
			Method: clickhousego.CompressionLZ4,
		},
	})

	_, err := conn.Exec("SELECT 1")
	if err != nil {
		return nil, err
	}
	return conn, nil
}

func testSetSetting(conn *sql.DB) error {
	// we can use context to pass settings to a specific API call
	ctx := clickhousego.Context(context.Background(), clickhousego.WithSettings(clickhousego.Settings{
		"async_insert": "1",
	}), clickhousego.WithProfileInfo(func(p *clickhousego.ProfileInfo) {
		fmt.Println("profile info: ", p)
	}))
	var settingValue bool
	if err := conn.QueryRowContext(ctx, "SELECT getSetting('async_insert')").Scan(&settingValue); err != nil {
		return fmt.Errorf("failed to get setting value: %v", err)
	}
	fmt.Printf("settingValue: %t\n", settingValue)
	if !settingValue {
		return fmt.Errorf("expected setting value to be true, got false")
	}
	return nil
}

Log

When working:

profile info:  rows=1, bytes=4104, blocks=1, rows before limit=0, applied limit=false, calculated rows before limit=true
settingValue: true

With the HTTP.Protocol:

settingValue: true

Details

There seem to be quite a few references that I've found in the repository for working with the Progress and Profile structs, but none with HTTP protocol. Putting them here in case it's helpful to anyone:

1. progress,profile, and log information docs
2. context
3. a test

Environment

• clickhouse-go version: v2.23.2
• Interface: ClickHouse API / database/sql compatible driver
• Go version: go version go1.24.0 linux/amd64
• Operating system:
• ClickHouse version: ClickHouse client version 24.3.5.47.altinitystable (altinity build)
• Is it a ClickHouse Cloud? - No
• ClickHouse Server non-default settings, if any:
• CREATE TABLE statements for tables involved:
• Sample data for all these tables, use clickhouse-obfuscator if necessary