Codestin Search App

shalapatil · 2022-06-01T05:34:25Z

Notion card

Summary

https://www.notion.so/saeloun/Back-end-Groupby-on-Reports-Filter-150697fe97e14036aec0400e20be8e17

Preview

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to
not work as expected)
This change requires a documentation update

How Has This Been Tested?

Checklist:

I have manually tested all workflows
I have performed a self-review of my own code
I have added automated tests for my code

keshavbiswa

LGTM! added minor comments.

keshavbiswa · 2022-06-01T05:57:22Z

app/services/report/group_by.rb

+    end
+
+    def process
+      return {} if group_by_field.blank? || !POSSIBLE_GROUP_BY_INPUTS.include?(group_by_field)


Suggested change

return {} if group_by_field.blank? || !POSSIBLE_GROUP_BY_INPUTS.include?(group_by_field)

return {} if group_by_field.blank? || POSSIBLE_GROUP_BY_INPUTS.exclude?(group_by_field)

keshavbiswa · 2022-06-01T05:59:16Z

app/services/report/result.rb

+        # When we query ES, we get all matching timesheet entries as response even when we pass aggregation query.
+        # Those timesheet entries contains all required association but not the ones in aggregated data.
+        # So, in order to avoid queries for associated records,
+        # creating map of id -> timesheet entries from the general ES response and
+        # then using it for building final data by merging that with ids found in aggregation query.


Can you add these comments outside the method?

rohitjoshixyz

Code LGTM
I liked how you have split the code into services, making it very clean and easy to grasp.

I understand the PR was complex and needed a lot of research but we would also want to have specs for each service you added and a request spec for the controller action to make the code foolproof.

Could you please also attach a loom video demonstrating the functionality?

Overall good job Shalaka. Well done 👏

xrimpy · 2022-06-02T09:53:07Z

app/models/timesheet_entry.rb


  scope :in_workspace, -> (company) { where(project_id: company&.project_ids) }

+  searchkick


For aggregations to work you have to have filterable is searchkick

searchable = %i[name] filterable = %[created_at] searchkick searchable: searchable, filterable: filterable

Not really. With aggregation query which I am passing, aggregations are working fine

xrimpy · 2022-06-02T09:53:50Z

app/models/timesheet_entry.rb


+  searchkick
+
+  def search_data


Can we individually list all the attributes we want to index?

For example

def search_data { id: id.to_i, name: name, created_at: created_at.to_datetime }

Reason for this is ES works on RAM, and unnecessary fields will clutter up the ES instance which becomes exponential over time.

Okay, I was thinking of doing this but thought that fields are not that much so indexed everything. I will update code to index only required ones

xrimpy · 2022-06-02T09:56:41Z

app/controllers/internal_api/v1/reports_controller.rb

      filters_where_clause = Report::Filters.process(params)
      where_clause = current_company_project_ids_filter.merge(filters_where_clause)
-      TimesheetEntry.search(where: where_clause, includes: [:user, :project])
+      group_by_clause = Report::GroupBy.process(params["group_by"])


Not sure how GroupBy works, for grouping you can use aggregations, as in the SearchKick documentation:

Product.search("wingtips", where: {color: "brandy"}, aggs: [:size])

using aggregation only which is defined in the service class

shalapatil

@rohitjoshixyz I can't add loom video as UI is broken and FE integration is required.

I am working on all the review comments now

rohitjoshixyz

Added few comments

rohitjoshixyz · 2022-06-06T08:30:47Z

spec/services/reports/group_by_spec.rb

+
+require "rails_helper"
+
+RSpec.describe "service::reports#group_by", type: :model do


type should be service

rohitjoshixyz · 2022-06-06T08:32:37Z

spec/requests/internal_api/v1/reports/index_spec.rb

+        expect(reports["label"]).to eq("")
+        timesheet_ids_in_response = reports["entries"].pluck("id")
+        expect(reports["entries"].size).to eq(2)
+        # Timsheets should be in desceding order of work_date


Assert in separate it block instead of a comment

rohitjoshixyz · 2022-06-06T08:37:54Z

spec/requests/internal_api/v1/reports/index_spec.rb

+        @this_week_start_date = 0.weeks.ago.beginning_of_week
+        @this_week_end_date = 0.weeks.ago.end_of_week
        @timesheet_entry1 = create(:timesheet_entry, project:, work_date: last_month_start_date)
-        @timesheet_entry2 = create(:timesheet_entry, project:, work_date: this_week_start_date)
-        @timesheet_entry3 = create(:timesheet_entry, project:, work_date: this_week_start_date)
+        @timesheet_entry2 = create(:timesheet_entry, project:, work_date: @this_week_start_date)
+        @timesheet_entry3 = create(:timesheet_entry, project:, work_date: @this_week_end_date)


Do not use @ variables, use let instead

rohitjoshixyz · 2022-06-06T08:41:11Z

spec/requests/internal_api/v1/reports/index_spec.rb

-        timesheet_ids_in_response = json_response["entries"].pluck("id")
-        expect(json_response["entries"].size).to eq(1)
+        reports = json_response["reports"].first
+        expect(reports["label"]).to eq("")


What are we asserting here?

we want label empty when we grouping is not selected

spec/requests/internal_api/v1/reports/index_spec.rb

keshavbiswa

Left some comments.

app/controllers/internal_api/v1/reports_controller.rb

spec/factories/timesheet_entries.rb

apoorv-mishra · 2022-06-06T11:44:37Z

app/services/report/group_by.rb

+  class GroupBy < ApplicationService
+    attr_reader :group_by_field
+
+    POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"]


Suggested change

POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"]

POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"].freeze

cc: @keshavbiswa @rohitjoshixyz

github-actions · 2022-06-06T12:19:23Z

Current Code Coverage Percent of this PR:

84.72 %

Files having coverage below 100%

Impacted Files	Coverage
/app/controllers/subscriptions_controller.rb	66.67 %
/app/controllers/invoices_controller.rb	57.14 %
/app/models/invoice.rb	96.3 %
/app/policies/client_policy.rb	95.45 %
/app/policies/invoice_policy.rb	94.12 %
/app/policies/subscriptions_policy.rb	66.67 %
/app/policies/timezone_policy.rb	66.67 %
/app/controllers/invoices/payments_controller.rb	61.11 %
/app/services/invoice_payment/checkout.rb	44.0 %
/app/services/invoice_payment/pdf_generation.rb	70.97 %
/app/services/report/filters.rb	87.1 %
/app/controllers/internal_api/v1/timezones_controller.rb	37.5 %
/app/controllers/internal_api/v1/clients_controller.rb	94.44 %
/app/controllers/internal_api/v1/invoices_controller.rb	94.87 %
/lib/benchmarking/benchmarker.rb	0.0 %
/lib/countries_info.rb	0.0 %

rohitjoshixyz

@shalapatil please add specs for filters and results service in a follow up PR so that our code coverage doesn't drop, other than that code LGTM. I am fine with keeping the @ variables for that one spec. cc @akhilgkrishnan @keshavbiswa .

Let's push this to production 🔥 if others do not object.

* Index client_id to elasticsearch * Use client id directly for filtering * Add service class for creating group-by es query * Add service class for processing aggregated ES data * Consider group-by while showing reports * Return formatted date labels for grouping by week * Add sorting as desc order by work_date * Add default filter of this_month duration * Fix reports request tests * Add specs for group_by service class * Add request specs for group by but without any filters * Add specs for group by and filter combinations * Fix clients specs * Fix suggested changes

shalapatil added 5 commits May 30, 2022 14:09

Index client_id to elasticsearch

671e300

Use client id directly for filtering

166f9d4

Add service class for creating group-by es query

29999a2

Add service class for processing aggregated ES data

25f8533

Consider group-by while showing reports

c83f9ed

pr-triage bot added the PR: unreviewed label Jun 1, 2022

shalapatil requested review from apoorv-mishra, keshavbiswa, rohitjoshixyz and xrimpy June 1, 2022 05:35

Return formatted date labels for grouping by week

608670a

keshavbiswa reviewed Jun 1, 2022

View reviewed changes

rohitjoshixyz reviewed Jun 1, 2022

View reviewed changes

shalapatil added 5 commits June 2, 2022 10:52

Add sorting as desc order by work_date

ce447a0

Add default filter of this_month duration

451a72e

Fix reports request tests

e596207

Add specs for group_by service class

0f39a50

Add request specs for group by but without any filters

5fe7d5c

xrimpy reviewed Jun 2, 2022

View reviewed changes

Add specs for group by and filter combinations

21b1b00

shalapatil requested review from keshavbiswa, rohitjoshixyz and xrimpy June 6, 2022 05:05

shalapatil commented Jun 6, 2022

View reviewed changes

rohitjoshixyz reviewed Jun 6, 2022

View reviewed changes

keshavbiswa reviewed Jun 6, 2022

View reviewed changes

app/controllers/internal_api/v1/reports_controller.rb Outdated Show resolved Hide resolved

spec/factories/timesheet_entries.rb Outdated Show resolved Hide resolved

apoorv-mishra reviewed Jun 6, 2022

View reviewed changes

shalapatil added 2 commits June 6, 2022 17:31

Fix clients specs

7697d61

Fix suggested changes

916fef6

rohitjoshixyz approved these changes Jun 7, 2022

View reviewed changes

pr-triage bot added PR: partially-approved and removed PR: unreviewed labels Jun 7, 2022

supriya3105 merged commit d0b8845 into develop Jun 7, 2022

supriya3105 deleted the group-reports branch June 7, 2022 08:11

pr-triage bot added PR: merged and removed PR: partially-approved labels Jun 7, 2022

	return {} if group_by_field.blank? \|\| !POSSIBLE_GROUP_BY_INPUTS.include?(group_by_field)
	return {} if group_by_field.blank? \|\| POSSIBLE_GROUP_BY_INPUTS.exclude?(group_by_field)


		scope :in_workspace, -> (company) { where(project_id: company&.project_ids) }

		searchkick


		require "rails_helper"

		RSpec.describe "service::reports#group_by", type: :model do

	POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"]
	POSSIBLE_GROUP_BY_INPUTS = ["team_member", "client", "project", "week"].freeze

Conversation

shalapatil commented Jun 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notion card

Summary

Preview

Type of change

How Has This Been Tested?

Checklist:

Uh oh!

keshavbiswa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rohitjoshixyz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shalapatil left a comment

Choose a reason for hiding this comment

Uh oh!

rohitjoshixyz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keshavbiswa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 6, 2022

Current Code Coverage Percent of this PR:

84.72 %

Files having coverage below 100%

Uh oh!

rohitjoshixyz left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

shalapatil commented Jun 1, 2022 •

edited

Loading