Getting query failure metrics as NaN always.

[msaha1994]

I am using the druid version of 0.12.

[elukey]

hi @msaha1994, I'd need a couple of info:

1. Are you using the last upstream version? If so, what config file are you using?
2. There is a debug option, can you try to use it and see if any error pops up?

[msaha1994]

I am not using the latest upstream.
Should I use sudo nohup .venv/bin/druid_exporter --debug --listen localhost:8000& to run in debug mode.
Also, I have one more question do I need to add recipient base URL config in common.runtime.properties to use this operations-software-druid-exporter?

[elukey]

Please do test the last upstream if possible :)

You need to instruct Druid to send metrics to the ip:port that the exporter is listening on, please check https://github.com/wikimedia/operations-software-druid_exporter#how-does-it-work.

[msaha1994]

I have taken the upstream pull. I am using the below command to run this.
sudo nohup .venv/bin/druid_exporter /home/ubuntu/operations-software-druid_exporter/conf/example_druid_0_12_3.json

My druid common.runtime.properties are-
druid.monitoring.monitors=["io.druid.java.util.metrics.SysMonitor","io.druid.java.util.metrics.JvmMonitor","io.druid.java.util.metrics.JvmCpuMonitor","io.druid.java.util.metrics.JvmThreadsMonitor","io.druid.server.metrics.EventReceiverFirehoseMonitor","io.druid.server.metrics.QueryCountStatsMonitor"]
druid.emitter=http
druid.monitoring.emissionPeriod=PT1m
druid.emitter.logging.logLevel=info
druid.emitter.http.recipientBaseUrl=http://10.10.18.254:8000/

Still not getting the data for query druid_broker_query_countt.
Do i need to chnage in collector.py?

[elukey]

Have you tried to query the brokers some times to see if datapoints are generated?

[msaha1994]

should i try curl http://localhost:8000/query/time/?

[elukey]

Nope what I mean is querying the druid broker itself, your example seems to point to the prometheus exporter. If you don't query the Druid broker no data point will be emitted and sent to the prometheus exporter, that will have no data to visualize.

[msaha1994]

My druid's recipient base URL port and operations-software-druid_exporter port both are the same. Will it be a problem?

[elukey]

Mmmm it is a bit strange, how could two different daemons (druid broker and prometheus exporter) bind to the same port on the same host? Are you sure?

[msaha1994]

So if I will set recipientBaseurl=http://10.10.40.198:8000/ and running exporter as sudo nohup .venv/bin/druid_exporter --listen 10.10.40.198:8000&
It should work??

[elukey]

Ah sorry I misread the above: yes please set recipientBaseurl in the Druid configuration to the address of the prometheus exporter. Then you have to query your Broker on port 8082 and see if datapoints appear (if it is not 8082 then the port that you configured for the broker).

[msaha1994]

I should try this curl http://10.10.18.254:8082/druid/broker/query/time right?

[elukey]

@msaha1994 you should try to query the Druid brokers as you would do to simulate any Druid workload, and then see if any datapoint is visualized. The flow is:

user-query -> Druid -> datapoint-via-http -> prometheus exporter <- query for datapoints