Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(api): add status to /mon/metrics#3500

Merged
yesnault merged 18 commits intomasterfrom
ye-metrics-status
Dec 2, 2018
Merged

feat(api): add status to /mon/metrics#3500
yesnault merged 18 commits intomasterfrom
ye-metrics-status

Conversation

@yesnault
Copy link
Member

@yesnault yesnault commented Oct 23, 2018 

status{component="Global/Status",instance="cdsinstance",status="AL"} 1
status{component="Global/Version",instance="cdsinstance",status="OK"} 1
status{component="Global/api",instance="cdsinstance",status="AL"} 3
status{component="Global/dbmigrate",instance="cdsinstance",status="OK"} 0
status{component="Global/doc",instance="cdsinstance",status="WARN"} 2
status{component="Global/elasticsearch",instance="cdsinstance",status="OK"} 0
status{component="Global/hatchery",instance="cdsinstance",status="AL"} 0
status{component="Global/hooks",instance="cdsinstance",status="OK"} 1
status{component="Global/repositories",instance="cdsinstance",status="OK"} 0
status{component="Global/vcs",instance="cdsinstance",status="AL"} 1
status{component="api_0/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_0/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_0/Database",instance="cdsinstance",status="OK"} 1
status{component="api_0/Event",instance="cdsinstance",status="OK"} 1
status{component="api_0/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_0/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_0/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_0/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_0/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_0/Scheduler",instance="cdsinstance",status="WARN"} 1
status{component="api_0/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_0/Time",instance="cdsinstance",status="OK"} 1
status{component="api_0/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_0/Version",instance="cdsinstance",status="OK"} 1
status{component="api_0/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="api_1/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_1/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_1/Database",instance="cdsinstance",status="OK"} 1
status{component="api_1/Event",instance="cdsinstance",status="OK"} 1
status{component="api_1/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_1/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_1/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_1/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_1/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_1/Scheduler",instance="cdsinstance",status="OK"} 1
status{component="api_1/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_1/Time",instance="cdsinstance",status="OK"} 1
status{component="api_1/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_1/Version",instance="cdsinstance",status="OK"} 1
status{component="api_1/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="api_2/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_2/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_2/Database",instance="cdsinstance",status="OK"} 1
status{component="api_2/Event",instance="cdsinstance",status="OK"} 1
status{component="api_2/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_2/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_2/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_2/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_2/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_2/Scheduler",instance="cdsinstance",status="OK"} 1
status{component="api_2/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_2/Time",instance="cdsinstance",status="OK"} 1
status{component="api_2/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_2/Version",instance="cdsinstance",status="OK"} 1
status{component="api_2/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="hooksLocal/Time",instance="cdsinstance",status="OK"} 1
status{component="hooksLocal/Uptime",instance="cdsinstance",status="OK"} 1
status{component="hooksLocal/Version",instance="cdsinstance",status="OK"} 1
status{component="sample-service/sample-service/sample-service",instance="cdsinstance",status="WARN"} 1
status{component="sample-service2/sample-service2/sample-service2",instance="cdsinstance",status="WARN"} 1
status{component="vcsLocal/Github-RateLimit",instance="cdsinstance",status="AL"} 0
status{component="vcsLocal/Github-RateLimitRemaining",instance="cdsinstance",status="OK"} 5000
status{component="vcsLocal/Github-RateLimitReset",instance="cdsinstance",status="AL"} 1
status{component="vcsLocal/Time",instance="cdsinstance",status="OK"} 1
status{component="vcsLocal/Uptime",instance="cdsinstance",status="OK"} 1
status{component="vcsLocal/Version",instance="cdsinstance",status="OK"} 1

Signed-off-by: Yvonnick Esnault [email protected]

fsamin
fsamin requested changes Oct 24, 2018
View reviewed changes
@yesnault yesnault force-pushed the ye-metrics-status branch 3 times, most recently from bf77cf5 to a133f52 Compare November 15, 2018 10:31
golangcibot
golangcibot reviewed Nov 15, 2018
View reviewed changes
fsamin
fsamin requested changes Nov 15, 2018
View reviewed changes
golangcibot
golangcibot reviewed Nov 15, 2018
View reviewed changes
golangcibot
golangcibot reviewed Nov 15, 2018
View reviewed changes
@yesnault
Copy link
Member Author

yesnault commented Nov 15, 2018

@fsamin after serie refactor:

nb_applications{cds="cdsinstance"} 93
nb_artifacts{cds="cdsinstance"} 0
nb_groups{cds="cdsinstance"} 703
nb_max_workers_building{cds="cdsinstance"} 0
nb_pipelines{cds="cdsinstance"} 266
nb_projects{cds="cdsinstance"} 228
nb_users{cds="cdsinstance"} 664
nb_worker_models{cds="cdsinstance"} 1
nb_workflow_node_runs{cds="cdsinstance"} 240354
nb_workflow_runs{cds="cdsinstance"} 36228
nb_workflows{cds="cdsinstance"} 102
queue{cds="cdsinstance",range="10_less_10s",status="waiting"} 0
queue{cds="cdsinstance",range="20_more_10s_less_30s",status="waiting"} 0
queue{cds="cdsinstance",range="30_more_30s_less_1min",status="waiting"} 0
queue{cds="cdsinstance",range="40_more_1min_less_2min",status="waiting"} 0
queue{cds="cdsinstance",range="50_more_2min_less_5min",status="waiting"} 0
queue{cds="cdsinstance",range="60_more_5min_less_10min",status="waiting"} 0
queue{cds="cdsinstance",range="70_more_10min",status="waiting"} 4
queue{cds="cdsinstance",range="all",status="building"} 0
status{cds="cdsinstance",name="Global",status="AL",type="global"} 1
status_api{cds="cdsinstance",name="Global",status="AL",type="global"} 2
status_cache{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 1
status_cache{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_database{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 1
status_database{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_dbmigrate{cds="cdsinstance",name="Global",status="OK",type="global"} 0
status_doc{cds="cdsinstance",name="Global",status="WARN",type="global"} 2
status_elasticsearch{cds="cdsinstance",name="Global",status="OK",type="global"} 0
status_event{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 1
status_event{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_github_ratelimit{cds="cdsinstance",name="vcsLocal",status="AL",type="vcs"} 0
status_github_ratelimitremaining{cds="cdsinstance",name="vcsLocal",status="OK",type="vcs"} 5000
status_github_ratelimitreset{cds="cdsinstance",name="vcsLocal",status="AL",type="vcs"} 1
status_hatchery{cds="cdsinstance",name="Global",status="AL",type="global"} 0
status_hooks{cds="cdsinstance",name="Global",status="OK",type="global"} 1
status_internal_events_queue{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 0
status_internal_events_queue{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 0
status_nb_of_panics{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 0
status_nb_of_panics{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 0
status_object_store{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 1
status_object_store{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_repositories{cds="cdsinstance",name="Global",status="OK",type="global"} 0
status_sample_service2_sample_service2{cds="cdsinstance",name="sample-service2",status="WARN",type="doc"} 1
status_sample_service_sample_service{cds="cdsinstance",name="sample-service",status="WARN",type="doc"} 1
status_scheduler{cds="cdsinstance",name="api_craandprajen",status="WARN",type="api"} 1
status_scheduler{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_sessions_store{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 1
status_sessions_store{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 1
status_smtp{cds="cdsinstance",name="api_craandprajen",status="AL",type="api"} 1
status_smtp{cds="cdsinstance",name="api_muskir",status="AL",type="api"} 1
status_vcs{cds="cdsinstance",name="Global",status="AL",type="global"} 1
status_worker_model_errors{cds="cdsinstance",name="api_craandprajen",status="OK",type="api"} 0
status_worker_model_errors{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 0

I can remove status_ if needed

@fsamin
Copy link
Member

fsamin commented Nov 19, 2018 •
edited
Loading

About statuses: as far as i understand, the number of "AL" and "OK" are important measures. Why not having (instead of status="AL" tag)

service_alarm{cds="cdsinstance",name="api_muskir", type="api"} 1
service_alarm{cds="cdsinstance",name="api_craandprajen", type="api"} 0
service_alarm{cds="cdsinstance",name="elasticsearch_foo", type="elasticsearchi"} 1
service_alarm{cds="cdsinstance",name="vcs_blabla",type="vcs"} 0

so we can aggregate per type, instance, or whatever the number of alarms

something like

status_nb_of_panics{cds="cdsinstance",name="api_muskir",status="OK",type="api"} 0

should not have status tag and be named nb_of_panics

stuff like

status{cds="cdsinstance",name="Global",status="AL",type="global"} 1
status_api{cds="cdsinstance",name="Global",status="AL",type="global"} 2
...

and all the measures with type="global"sound like aggregate measures and have no sense, because aggregation must be done by the consumer of the metrics, not the producer.

golangcibot
golangcibot reviewed Nov 20, 2018
View reviewed changes
@yesnault
feat(api): add status to /mon/metrics
f93447b 
```
status{component="Global/Status",instance="cdsinstance",status="AL"} 1
status{component="Global/Version",instance="cdsinstance",status="OK"} 1
status{component="Global/api",instance="cdsinstance",status="AL"} 3
status{component="Global/dbmigrate",instance="cdsinstance",status="OK"} 0
status{component="Global/doc",instance="cdsinstance",status="WARN"} 2
status{component="Global/elasticsearch",instance="cdsinstance",status="OK"} 0
status{component="Global/hatchery",instance="cdsinstance",status="AL"} 0
status{component="Global/hooks",instance="cdsinstance",status="OK"} 1
status{component="Global/repositories",instance="cdsinstance",status="OK"} 0
status{component="Global/vcs",instance="cdsinstance",status="AL"} 1
status{component="api_0/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_0/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_0/Database",instance="cdsinstance",status="OK"} 1
status{component="api_0/Event",instance="cdsinstance",status="OK"} 1
status{component="api_0/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_0/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_0/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_0/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_0/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_0/Scheduler",instance="cdsinstance",status="WARN"} 1
status{component="api_0/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_0/Time",instance="cdsinstance",status="OK"} 1
status{component="api_0/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_0/Version",instance="cdsinstance",status="OK"} 1
status{component="api_0/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="api_1/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_1/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_1/Database",instance="cdsinstance",status="OK"} 1
status{component="api_1/Event",instance="cdsinstance",status="OK"} 1
status{component="api_1/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_1/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_1/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_1/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_1/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_1/Scheduler",instance="cdsinstance",status="OK"} 1
status{component="api_1/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_1/Time",instance="cdsinstance",status="OK"} 1
status{component="api_1/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_1/Version",instance="cdsinstance",status="OK"} 1
status{component="api_1/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="api_2/CDSName",instance="cdsinstance",status="OK"} 1
status{component="api_2/Cache",instance="cdsinstance",status="OK"} 1
status{component="api_2/Database",instance="cdsinstance",status="OK"} 1
status{component="api_2/Event",instance="cdsinstance",status="OK"} 1
status{component="api_2/Hostname",instance="cdsinstance",status="OK"} 1
status{component="api_2/Internal Events Queue",instance="cdsinstance",status="OK"} 0
status{component="api_2/Nb of Panics",instance="cdsinstance",status="OK"} 0
status{component="api_2/Object-Store",instance="cdsinstance",status="OK"} 1
status{component="api_2/SMTP",instance="cdsinstance",status="AL"} 1
status{component="api_2/Scheduler",instance="cdsinstance",status="OK"} 1
status{component="api_2/Sessions-Store",instance="cdsinstance",status="OK"} 1
status{component="api_2/Time",instance="cdsinstance",status="OK"} 1
status{component="api_2/Uptime",instance="cdsinstance",status="OK"} 1
status{component="api_2/Version",instance="cdsinstance",status="OK"} 1
status{component="api_2/Worker Model Errors",instance="cdsinstance",status="OK"} 0
status{component="hooksLocal/Time",instance="cdsinstance",status="OK"} 1
status{component="hooksLocal/Uptime",instance="cdsinstance",status="OK"} 1
status{component="hooksLocal/Version",instance="cdsinstance",status="OK"} 1
status{component="sample-service/sample-service/sample-service",instance="cdsinstance",status="WARN"} 1
status{component="sample-service2/sample-service2/sample-service2",instance="cdsinstance",status="WARN"} 1
status{component="vcsLocal/Github-RateLimit",instance="cdsinstance",status="AL"} 0
status{component="vcsLocal/Github-RateLimitRemaining",instance="cdsinstance",status="OK"} 5000
status{component="vcsLocal/Github-RateLimitReset",instance="cdsinstance",status="AL"} 1
status{component="vcsLocal/Time",instance="cdsinstance",status="OK"} 1
status{component="vcsLocal/Uptime",instance="cdsinstance",status="OK"} 1
status{component="vcsLocal/Version",instance="cdsinstance",status="OK"} 1
```

Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
comment
172e86b 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
799f0a7 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
a43e3fb 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
f23e250 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
7a83562 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
425fc4b 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
b91f2a9 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
b83c35e 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
ad8f807 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
newView everywhere
9314c21 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
0b4b41d 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
log if err
04474cd 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr: remove global status, remove prefix
109d0f1 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
fix merge
c85b258 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
we need global for alert on minInstances
9251e2b 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
3ff7151 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault
cr
5090987 
Signed-off-by: Yvonnick Esnault <[email protected]>
@yesnault yesnault removed the refactor label Nov 23, 2018
@yesnault yesnault merged commit 9ca2976 into master Dec 2, 2018
@yesnault yesnault deleted the ye-metrics-status branch December 15, 2018 22:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fsamin fsamin fsamin approved these changes

@sguiheux sguiheux sguiheux approved these changes

@richardlt richardlt richardlt approved these changes

+2 more reviewers

@golangcibot golangcibot golangcibot left review comments

@bnjjj bnjjj bnjjj approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@yesnault @fsamin @sguiheux @richardlt @bnjjj @golangcibot