Cache label IDs #1414
Cache label IDs #1414
Conversation
ce7ae00 to
23147d4
Compare
|
Closes #1392 |
|
@niksajakovljevic have you considered using (label pair)=> (id, map[metric_name]=>pos) instead? That would allow reusing the existing cache and thus improve overall cache hit ratio? |
23147d4 to
7377014
Compare
Labels reader cache is actually inverse |
ab8d180 to
91e629c
Compare
| for _, cachedLabel := range info.cachedLabels { | ||
| if val, ok := labelMap[cachedLabel]; ok { | ||
| if int(val.Pos) > info.maxPos { | ||
| info.maxPos = int(val.Pos) |
There was a problem hiding this comment.
Can we do this when fetching from cache?
There was a problem hiding this comment.
Yeah we have to do it. Since some labels are cached and we need maxPos to be correct (meaning it contains the max for both fetched and cached labels)
There was a problem hiding this comment.
No, I meant, can you start calculating the max position when fetching cached entries, so you don't have to iterate again through the cached labels.
91e629c to
49ca9a7
Compare
49ca9a7 to
fafbd69
Compare
Add inverted cache ( (metric + label pair) -> (id, pos)) to avoid DB calls for fetching label IDs in cases when series ID is not cached. This cache is only used when ingesting data. Benchmarks are showing around 5-10% gains in ingest performance and about 25% less DB calls for fetching label IDs (note that these numbers depend a lot on a shape of the dataset).
fafbd69 to
2396c0e
Compare
Add inverted cache ( (metric + label pair) -> (id, pos)) to avoid DB calls for
fetching label IDs in cases when series ID is not cached.
This cache is only used on metric ingestion.
Benchmarks are showing around 5-10% gains in ingest performance and
about 25% less DB calls for fetching label IDs (note that these numbers
depend a lot on a shape of the dataset).