-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Implementation/COMMS-1046: Labels in the work package schema and labels-by-workspace API #25552
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
e20f90b
8812c49
ebcdc56
ab4d4e0
1993a75
b6b5582
8ab6517
389bd60
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -32,8 +32,24 @@ class Label < ApplicationRecord | |||||||||||||||||||||||||||||||||
| belongs_to :author, class_name: "User" | ||||||||||||||||||||||||||||||||||
| has_many :labelings, dependent: :delete_all | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| USAGE_COUNT_SQL = "(SELECT COUNT(*) FROM labelings WHERE labelings.label_id = labels.id)" | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| scope :with_usage_count, -> { | ||||||||||||||||||||||||||||||||||
| select("labels.*, (SELECT COUNT(*) FROM labelings WHERE labelings.label_id = labels.id) AS usage_count") | ||||||||||||||||||||||||||||||||||
| select("labels.*, #{USAGE_COUNT_SQL} AS usage_count") | ||||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| scope :ordered_by_relevance_for, ->(project) { | ||||||||||||||||||||||||||||||||||
| used_in_project = Labeling | ||||||||||||||||||||||||||||||||||
| .where(labelable_type: WorkPackage.name) | ||||||||||||||||||||||||||||||||||
| .where("labelings.label_id = labels.id") | ||||||||||||||||||||||||||||||||||
| .joins("INNER JOIN work_packages ON work_packages.id = labelings.labelable_id") | ||||||||||||||||||||||||||||||||||
| .where(work_packages: { project_id: project }) | ||||||||||||||||||||||||||||||||||
| .arel | ||||||||||||||||||||||||||||||||||
| .exists | ||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||
| reorder(used_in_project.desc) | ||||||||||||||||||||||||||||||||||
| .order(Arel.sql("#{USAGE_COUNT_SQL} DESC")) | ||||||||||||||||||||||||||||||||||
| .order("LOWER(labels.name) ASC") | ||||||||||||||||||||||||||||||||||
|
Comment on lines
+41
to
+52
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 Benchmark:
|
||||||||||||||||||||||||||||||||||
| Scenario (labels / labelings / WPs) | Unfiltered | Filtered | total count |
Alphabetical baseline |
|---|---|---|---|---|
| Small (100 / 10k / 5k) | 4.7 / 68.7 | 0.7 / 1.9 | 0.3 | 1.9 |
| Medium (1k / 100k / 50k) | 97.2 / 114.5 | 1.2 / 2.6 | 0.3 | 31.6 |
| Large (10k / 1M / 500k) | 627.5 / 841.8 | 5.0 / 5.9 | 0.6 | 274.2 |
Where the time goes. The "used in this project" EXISTS is cheap: Postgres turns it into a hashed subplan that runs once (loops=1). The cost is the correlated usage_count subquery, which runs once per label (loops=10000, ~1M buffer hits at the large size). The name filter is applied before those subqueries are evaluated, so the per-keystroke filtered path stays in single-digit ms even at the large size. Only the first, unfiltered open of the dropdown gets slow, and only on instances with thousands of labels and ~1M labelings.
Alternatives tried (large size only):
| Rewrite | Unfiltered | Filtered |
|---|---|---|
EXISTS as a CTE |
~623 | not run |
usage_count via one GROUP BY |
~227 | not run |
| Both combined | ~90 | ~87 |
The combined rewrite returns the same order and brings the unfiltered open under 100ms, but it makes every filtered keystroke ~17x slower (5ms to 87ms), because aggregating over all labelings is a fixed cost the name filter can't reduce.
Decision: keeping the current query. The keystroke path is fast at every size, and the slow case is a first open on very large instances. If that becomes a real problem, the GROUP BY rewrite can be applied to the unfiltered request only.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice. This gives confidence. Thanks for making it.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| #-- copyright | ||
| # OpenProject is an open source project management software. | ||
| # Copyright (C) the OpenProject GmbH | ||
| # | ||
| # This program is free software; you can redistribute it and/or | ||
| # modify it under the terms of the GNU General Public License version 3. | ||
| # | ||
| # OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows: | ||
| # Copyright (C) 2006-2013 Jean-Philippe Lang | ||
| # Copyright (C) 2010-2013 the ChiliProject Team | ||
| # | ||
| # This program is free software; you can redistribute it and/or | ||
| # modify it under the terms of the GNU General Public License | ||
| # as published by the Free Software Foundation; either version 2 | ||
| # of the License, or (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program; if not, write to the Free Software | ||
| # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. | ||
| # | ||
| # See COPYRIGHT and LICENSE files for more details. | ||
| #++ | ||
|
|
||
| module WorkPackage::Exports | ||
| module Formatters | ||
| class Labels < ::Exports::Formatters::Default | ||
| def self.apply?(attribute, _export_format) | ||
| attribute.to_sym == :labels | ||
| end | ||
|
|
||
| def retrieve_value(object) | ||
| object.labels.map(&:name) | ||
| end | ||
| end | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| # /api/v3/workspaces/{id}/labels | ||
| --- | ||
| get: | ||
| parameters: | ||
| - description: ID of the workspace whose labels will be listed | ||
| example: 1 | ||
| in: path | ||
| name: id | ||
| required: true | ||
| schema: | ||
| type: integer | ||
| - name: filters | ||
| description: |- | ||
| JSON specifying filter conditions. | ||
| Currently supported filters are: | ||
|
|
||
| + name: filters labels by name, with the operators `~` (contains), `!~` (does not contain) and `**` (all) | ||
| example: '[{ "name": { "operator": "~", "values": ["bug"] } }]' | ||
| in: query | ||
| required: false | ||
| schema: | ||
| type: string | ||
| responses: | ||
| '200': | ||
| content: | ||
| application/hal+json: | ||
| examples: | ||
| 'simple label collection': | ||
| $ref: "../components/examples/label_collection.yml" | ||
| schema: | ||
| "$ref": "../components/schemas/label_collection_model.yml" | ||
| description: OK | ||
| headers: {} | ||
| '400': | ||
| $ref: "../components/responses/invalid_query.yml" | ||
| '404': | ||
| content: | ||
| application/hal+json: | ||
| schema: | ||
| $ref: "../components/schemas/error_response.yml" | ||
| examples: | ||
| response: | ||
| value: | ||
| _type: Error | ||
| errorIdentifier: urn:openproject-org:api:v3:errors:NotFound | ||
| message: The specified workspace does not exist. | ||
| description: |- | ||
| Returned if the workspace does not exist or the client does not have sufficient permissions | ||
| to see it. | ||
|
|
||
| **Required permission:** view work packages (on given workspace) | ||
|
|
||
| *Note: A client without sufficient permissions shall not be able to test for the existence of a workspace. | ||
| That's why a 404 is returned here, even if a 403 might be more appropriate.* | ||
| headers: {} | ||
| tags: | ||
| - Labels | ||
| description: |- | ||
| Returns a paginated collection of all labels, ordered by relevance for the given workspace: | ||
| labels already used on work packages of the workspace come first, followed by the remaining labels. | ||
| Within each group, labels are ordered by how often they are used overall and then by name. | ||
| operationId: List_labels_by_workspace | ||
| summary: List labels by workspace |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| #-- copyright | ||
| # OpenProject is an open source project management software. | ||
| # Copyright (C) the OpenProject GmbH | ||
| # | ||
| # This program is free software; you can redistribute it and/or | ||
| # modify it under the terms of the GNU General Public License version 3. | ||
| # | ||
| # OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows: | ||
| # Copyright (C) 2006-2013 Jean-Philippe Lang | ||
| # Copyright (C) 2010-2013 the ChiliProject Team | ||
| # | ||
| # This program is free software; you can redistribute it and/or | ||
| # modify it under the terms of the GNU General Public License | ||
| # as published by the Free Software Foundation; either version 2 | ||
| # of the License, or (at your option) any later version. | ||
| # | ||
| # This program is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program; if not, write to the Free Software | ||
| # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. | ||
| # | ||
| # See COPYRIGHT and LICENSE files for more details. | ||
| #++ | ||
|
|
||
| module API | ||
| module V3 | ||
| module Labels | ||
| class LabelsByWorkspaceAPI < ::API::OpenProjectAPI | ||
| resources :labels do | ||
| after_validation do | ||
| raise API::Errors::NotFound unless OpenProject::FeatureDecisions.work_package_labels_active? | ||
|
|
||
| authorize_in_project(:view_work_packages, project: @project) | ||
| end | ||
|
|
||
| get &::API::V3::Utilities::Endpoints::Index.new(model: Label, | ||
| scope: -> { Label.ordered_by_relevance_for(@project) }, | ||
| self_path: -> { api_v3_paths.labels_by_workspace(@project.id) }) | ||
| .mount | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟢 I am debating this with myself, but I feel there's something a bit off
relevanceis ambiguous, it can mean anything and it does not explain what the sorting is doingFor me either add an explanation comment...
... or make the scope name overexplain itself
but if this changes in the future we need to change the method signature 👀