-
Notifications
You must be signed in to change notification settings - Fork 88
Group reports #429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Group reports #429
Changes from all commits
671e300
166f9d4
29999a2
25f8533
c83f9ed
608670a
ce447a0
451a72e
e596207
0f39a50
5fe7d5c
21b1b00
7697d61
916fef6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,8 +26,6 @@ | |
| # | ||
|
|
||
| class TimesheetEntry < ApplicationRecord | ||
| searchkick | ||
|
|
||
| enum bill_status: [:non_billable, :unbilled, :billed] | ||
|
|
||
| belongs_to :user | ||
|
|
@@ -44,6 +42,19 @@ class TimesheetEntry < ApplicationRecord | |
|
|
||
| scope :in_workspace, -> (company) { where(project_id: company&.project_ids) } | ||
|
|
||
| searchkick | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For aggregations to work you have to have
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not really. With aggregation query which I am passing, aggregations are working fine |
||
|
|
||
| def search_data | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we individually list all the attributes we want to index? For example Reason for this is ES works on RAM, and unnecessary fields will clutter up the ES instance which becomes exponential over time.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, I was thinking of doing this but thought that fields are not that much so indexed everything. I will update code to index only required ones |
||
| { | ||
| id: id.to_i, | ||
| bill_status:, | ||
| project_id:, | ||
| client_id: self.project.client_id, | ||
| user_id:, | ||
| work_date: | ||
| } | ||
| end | ||
|
|
||
| def self.during(from, to) | ||
| where(work_date: from..to).order(work_date: :desc) | ||
| end | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module Report | ||
| class GroupBy < ApplicationService | ||
| attr_reader :group_by_field | ||
|
|
||
| POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"].freeze | ||
| GROUP_BY_INPUT_TO_ES_FIELD = { | ||
| "team_member" => "user_id", | ||
| "client" => "client_id", | ||
| "project" => "project_id", | ||
| "week" => "week" | ||
| } | ||
|
|
||
| def initialize(group_by_field) | ||
| @group_by_field = group_by_field | ||
| end | ||
|
|
||
| def process | ||
| return {} if group_by_field.blank? || POSSIBLE_GROUP_BY_INPUTS.exclude?(group_by_field) | ||
|
|
||
| group_by_query(GROUP_BY_INPUT_TO_ES_FIELD[group_by_field]) | ||
| end | ||
|
|
||
| def group_by_query(field) | ||
| { | ||
| aggs: { | ||
| grouped_reports: group_by_term(field).merge( | ||
| { | ||
| aggs: { | ||
| top_report_hits: { | ||
| top_hits: { | ||
| sort: [{ work_date: { order: "desc" } }], | ||
| _source: ["id"], | ||
| size: 100 # aggregation query can only return max top 100 hits | ||
| } | ||
| } | ||
| } | ||
| }) | ||
| } | ||
| } | ||
| end | ||
|
|
||
| def group_by_term(field) | ||
| if field == "week" | ||
| { date_histogram: { field: "work_date", calendar_interval: "week" } } | ||
| else | ||
| { terms: { field: } } | ||
| end | ||
| end | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module Report | ||
| class Result < ApplicationService | ||
| attr_reader :es_response, :group_by | ||
|
|
||
| def initialize(es_response, group_by) | ||
| @es_response = es_response | ||
| @group_by = group_by | ||
| end | ||
|
|
||
| def process | ||
| if group_by.blank? | ||
| [{ label: "", entries: es_response }] | ||
| else | ||
| process_aggregated_es_response | ||
| end | ||
| end | ||
|
|
||
| private | ||
|
|
||
| # When we query ES, we get all matching timesheet entries as response even when we pass aggregation query. | ||
| # Those timesheet entries contains all required association but not the ones in aggregated data. | ||
| # So, in order to avoid queries for associated records, | ||
| # creating map of id -> timesheet entries from the general ES response and | ||
| # then using it for building final data by merging that with ids found in aggregation query. | ||
| def process_aggregated_es_response | ||
| id_to_timesheet_entry = timsheet_id_to_timesheet_entry | ||
| buckets = es_response.aggs["grouped_reports"]["buckets"] | ||
| buckets.map do |bucket| | ||
| timesheet_entry_ids = bucket["top_report_hits"]["hits"]["hits"].pluck("_id") | ||
| timesheet_entries = id_to_timesheet_entry.fetch_values(*timesheet_entry_ids) | ||
| { | ||
| label: group_label(timesheet_entries.first, bucket["key_as_string"]), | ||
| entries: timesheet_entries | ||
| } | ||
| end | ||
| end | ||
|
|
||
| def timsheet_id_to_timesheet_entry | ||
| es_response.reduce({}) do |res, timesheet| | ||
| res.merge({ timesheet.id.to_s => timesheet }) | ||
| end | ||
| end | ||
|
|
||
| def group_label(timesheet_entry, bucket_name) | ||
| case group_by | ||
| when "team_member" | ||
| timesheet_entry.user.full_name | ||
| when "client" | ||
| timesheet_entry.project.client.name | ||
| when "project" | ||
| timesheet_entry.project.name | ||
| when "week" | ||
| start_date = DateTime.parse(bucket_name) | ||
| end_date = start_date + 6.days | ||
| date_format = "%d %b %Y" | ||
| "#{start_date.strftime(date_format)} - #{end_date.strftime(date_format)}" | ||
| end | ||
| end | ||
| end | ||
| end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure how GroupBy works, for grouping you can use aggregations, as in the SearchKick documentation:
Product.search("wingtips", where: {color: "brandy"}, aggs: [:size])There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using aggregation only which is defined in the service class