Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion app/models/label.rb
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,24 @@ class Label < ApplicationRecord
belongs_to :author, class_name: "User"
has_many :labelings, dependent: :delete_all

USAGE_COUNT_SQL = "(SELECT COUNT(*) FROM labelings WHERE labelings.label_id = labels.id)"

scope :with_usage_count, -> {
select("labels.*, (SELECT COUNT(*) FROM labelings WHERE labelings.label_id = labels.id) AS usage_count")
select("labels.*, #{USAGE_COUNT_SQL} AS usage_count")
}

scope :ordered_by_relevance_for, ->(project) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 I am debating this with myself, but I feel there's something a bit off

  • relevance is ambiguous, it can mean anything and it does not explain what the sorting is doing
  • reading the code explains it, but it takes a minute to parse everything
  • the product solution is not really tested and might change in the future

For me either add an explanation comment...

# Sorts first by locality (labels already used in the current project come first)
# then by frequency (most used first).

... or make the scope name overexplain itself

scope :ordered_by_locality_and_usage_for, -> (project) {

but if this changes in the future we need to change the method signature 👀

used_in_project = Labeling
.where(labelable_type: WorkPackage.name)
.where("labelings.label_id = labels.id")
.joins("INNER JOIN work_packages ON work_packages.id = labelings.labelable_id")
.where(work_packages: { project_id: project })
.arel
.exists

reorder(used_in_project.desc)
.order(Arel.sql("#{USAGE_COUNT_SQL} DESC"))
.order("LOWER(labels.name) ASC")
Comment on lines +41 to +52

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Benchmark: Label.ordered_by_relevance_for

The labels dropdown calls this endpoint with pageSize=-1 (clamped to apiv3_max_page_size, 1000 by default) and re-queries with a name ~ filter on each keystroke, so I benchmarked the exact SQL the index endpoint runs at three data sizes. Data was bulk-inserted with a skewed labeling distribution, the target project holding ~1-2% of work packages. Timings are server-side query times in ms (median / p95 over 19 warm runs).

Scenario (labels / labelings / WPs) Unfiltered Filtered total count Alphabetical baseline
Small (100 / 10k / 5k) 4.7 / 68.7 0.7 / 1.9 0.3 1.9
Medium (1k / 100k / 50k) 97.2 / 114.5 1.2 / 2.6 0.3 31.6
Large (10k / 1M / 500k) 627.5 / 841.8 5.0 / 5.9 0.6 274.2

Where the time goes. The "used in this project" EXISTS is cheap: Postgres turns it into a hashed subplan that runs once (loops=1). The cost is the correlated usage_count subquery, which runs once per label (loops=10000, ~1M buffer hits at the large size). The name filter is applied before those subqueries are evaluated, so the per-keystroke filtered path stays in single-digit ms even at the large size. Only the first, unfiltered open of the dropdown gets slow, and only on instances with thousands of labels and ~1M labelings.

Alternatives tried (large size only):

Rewrite Unfiltered Filtered
EXISTS as a CTE ~623 not run
usage_count via one GROUP BY ~227 not run
Both combined ~90 ~87

The combined rewrite returns the same order and brings the unfiltered open under 100ms, but it makes every filtered keystroke ~17x slower (5ms to 87ms), because aggregating over all labelings is a fixed cost the name filter can't reduce.

Decision: keeping the current query. The keystroke path is fast at every size, and the slow case is a first open on very large instances. If that becomes a real problem, the GROUP BY rewrite can be applied to the unfiltered request only.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. This gives confidence. Thanks for making it.

}

normalizes :name, with: -> { it.squish }
Expand Down
1 change: 1 addition & 0 deletions app/models/type/attribute_groups.rb
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ module Type::AttributeGroups
remaining_time: :estimates_and_progress,
percentage_done: :estimates_and_progress,
priority: :details,
labels: :details,
# `:excluded` is not a "real" group. It's meant to exclude built in fields from the form
observed_in_versions: :excluded
}
Expand Down
7 changes: 6 additions & 1 deletion app/models/type/attributes.rb
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ def all_work_package_form_attributes(merge_date: false)
OpenProject::Cache.fetch_request_cached("all_work_package_form_attributes",
*wp_cf_cache_parts,
EXCLUDED.length,
merge_date) do
merge_date,
OpenProject::FeatureDecisions.work_package_labels_active?) do
calculate_all_work_package_form_attributes(merge_date)
end
end
Expand Down Expand Up @@ -139,6 +140,10 @@ def skipped_attribute?(key, definition)
# We always want to include the priority even if its required
return false if key == "priority"

# Remove once the work_package_labels feature flag is removed; show_if
# on the schema representer property is never evaluated here.
return true if key == "labels" && !OpenProject::FeatureDecisions.work_package_labels_active?

EXCLUDED.include?(key) || definition[:required]
end

Expand Down
43 changes: 43 additions & 0 deletions app/models/work_package/exports/formatters/labels.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# frozen_string_literal: true

#-- copyright
# OpenProject is an open source project management software.
# Copyright (C) the OpenProject GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version 3.
#
# OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows:
# Copyright (C) 2006-2013 Jean-Philippe Lang
# Copyright (C) 2010-2013 the ChiliProject Team
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# See COPYRIGHT and LICENSE files for more details.
#++

module WorkPackage::Exports
module Formatters
class Labels < ::Exports::Formatters::Default
def self.apply?(attribute, _export_format)
attribute.to_sym == :labels
end

def retrieve_value(object)
object.labels.map(&:name)
end
end
end
end
1 change: 1 addition & 0 deletions config/initializers/export_formats.rb
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
formatter WorkPackage, WorkPackage::Exports::Formatters::XLS::DoneRatio
formatter WorkPackage, WorkPackage::Exports::Formatters::PDF::Hours
formatter WorkPackage, WorkPackage::Exports::Formatters::Id
formatter WorkPackage, WorkPackage::Exports::Formatters::Labels
formatter WorkPackage, WorkPackage::Exports::Formatters::ProjectPhase
formatter WorkPackage, WorkPackage::Exports::Formatters::SpentUnits
formatter WorkPackage, WorkPackage::Exports::Formatters::TargetVersions
Expand Down
2 changes: 2 additions & 0 deletions docs/api/apiv3/openapi-spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -665,6 +665,8 @@ paths:
"$ref": "./paths/workspace_work_packages_form.yml"
"/api/v3/workspaces/{id}/versions":
"$ref": "./paths/workspace_versions.yml"
"/api/v3/workspaces/{id}/labels":
"$ref": "./paths/workspace_labels.yml"
"/api/v3/workspaces/schema":
"$ref": "./paths/workspaces_schema.yml"

Expand Down
63 changes: 63 additions & 0 deletions docs/api/apiv3/paths/workspace_labels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# /api/v3/workspaces/{id}/labels
---
get:
parameters:
- description: ID of the workspace whose labels will be listed
example: 1
in: path
name: id
required: true
schema:
type: integer
- name: filters
description: |-
JSON specifying filter conditions.
Currently supported filters are:

+ name: filters labels by name, with the operators `~` (contains), `!~` (does not contain) and `**` (all)
example: '[{ "name": { "operator": "~", "values": ["bug"] } }]'
in: query
required: false
schema:
type: string
responses:
'200':
content:
application/hal+json:
examples:
'simple label collection':
$ref: "../components/examples/label_collection.yml"
schema:
"$ref": "../components/schemas/label_collection_model.yml"
description: OK
headers: {}
'400':
$ref: "../components/responses/invalid_query.yml"
'404':
content:
application/hal+json:
schema:
$ref: "../components/schemas/error_response.yml"
examples:
response:
value:
_type: Error
errorIdentifier: urn:openproject-org:api:v3:errors:NotFound
message: The specified workspace does not exist.
description: |-
Returned if the workspace does not exist or the client does not have sufficient permissions
to see it.

**Required permission:** view work packages (on given workspace)

*Note: A client without sufficient permissions shall not be able to test for the existence of a workspace.
That's why a 404 is returned here, even if a 403 might be more appropriate.*
headers: {}
tags:
- Labels
description: |-
Returns a paginated collection of all labels, ordered by relevance for the given workspace:
labels already used on work packages of the workspace come first, followed by the remaining labels.
Within each group, labels are ordered by how often they are used overall and then by name.
operationId: List_labels_by_workspace
summary: List labels by workspace
50 changes: 50 additions & 0 deletions lib/api/v3/labels/labels_by_workspace_api.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# frozen_string_literal: true

#-- copyright
# OpenProject is an open source project management software.
# Copyright (C) the OpenProject GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version 3.
#
# OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows:
# Copyright (C) 2006-2013 Jean-Philippe Lang
# Copyright (C) 2010-2013 the ChiliProject Team
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# See COPYRIGHT and LICENSE files for more details.
#++

module API
module V3
module Labels
class LabelsByWorkspaceAPI < ::API::OpenProjectAPI
resources :labels do
after_validation do
raise API::Errors::NotFound unless OpenProject::FeatureDecisions.work_package_labels_active?

authorize_in_project(:view_work_packages, project: @project)
end

get &::API::V3::Utilities::Endpoints::Index.new(model: Label,
scope: -> { Label.ordered_by_relevance_for(@project) },
self_path: -> { api_v3_paths.labels_by_workspace(@project.id) })
.mount
end
end
end
end
end
4 changes: 4 additions & 0 deletions lib/api/v3/utilities/path_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,10 @@ def self.views_type(type)
index :label
show :label

def self.labels_by_workspace(workspace_id)
"#{workspace(workspace_id)}/labels"
end

def self.versions_available_projects
"#{versions}/available_projects"
end
Expand Down
16 changes: 12 additions & 4 deletions lib/api/v3/work_packages/eager_loading/checksum.rb
Original file line number Diff line number Diff line change
Expand Up @@ -56,24 +56,32 @@ def fetch_checksums_for(work_packages)

protected

# Versions are a has_many, which would multiply rows in the
# left_joins/pluck above, so they enter as an aggregated subquery.
# A version can attach under more than one kind, so the kind is part
# of the value and of the order.
# Versions and labels are has_many and enter as aggregated subqueries;
# joining them in the pluck above would multiply the rows. A version
# can attach under more than one kind, so the kind is part of the
# value and of the order.
VERSIONS_CHECKSUM_SQL = <<~SQL.squish
(SELECT COALESCE(STRING_AGG(CONCAT(wpv.kind, v.id, v.updated_at), ',' ORDER BY wpv.kind, v.id), '')
FROM work_package_versions wpv
INNER JOIN versions v ON v.id = wpv.version_id
WHERE wpv.work_package_id = work_packages.id)
SQL

LABELS_CHECKSUM_SQL = <<~SQL.squish
(SELECT COALESCE(STRING_AGG(CONCAT(l.id, l.updated_at), ',' ORDER BY l.id), '')
FROM labelings lg
INNER JOIN labels l ON l.id = lg.label_id
WHERE lg.labelable_type = '#{WorkPackage.polymorphic_name}' AND lg.labelable_id = work_packages.id)
SQL

def md5_concat
md5_parts = checksum_associations.flat_map do |association_name|
table_name = md5_checksum_table_name(association_name)

%W[#{table_name}.id #{table_name}.updated_at]
end
md5_parts << VERSIONS_CHECKSUM_SQL
md5_parts << LABELS_CHECKSUM_SQL

<<-SQL
MD5(CONCAT(#{md5_parts.join(', ')}))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ class WorkPackageSchemaRepresenter < ::API::Decorators::SchemaRepresenter
dependencies: -> {
all_permissions_granted_to_user_under_project +
[Setting.work_package_done_ratio,
Setting::WorkPackageMultipleVersions.active?]
Setting::WorkPackageMultipleVersions.active?,
OpenProject::FeatureDecisions.work_package_labels_active?]
}

custom_field_injector type: :schema_representer
Expand Down Expand Up @@ -279,6 +280,12 @@ def initialize(schema, self_link:, **context)
required: false,
href_callback: ->(*) { assignee_user_autocompleter }

schema_with_allowed_link :labels,
type: "[]Label",
required: false,
show_if: ->(*) { OpenProject::FeatureDecisions.work_package_labels_active? },
href_callback: ->(*) { labels_autocompleter }

schema_with_allowed_collection :type,
value_representer: Types::TypeRepresenter,
link_factory: ->(type) {
Expand Down Expand Up @@ -460,6 +467,12 @@ def all_permissions_granted_to_user_under_project
.sort
end

def labels_autocompleter
project_id = represented.work_package&.project_id

api_v3_paths.labels_by_workspace(project_id) if project_id
end

def assignee_user_autocompleter
work_package = represented.work_package

Expand Down
1 change: 1 addition & 0 deletions lib/api/v3/workspaces/nested_apis.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ class NestedApis < ::API::OpenProjectAPI
mount API::V3::WorkPackages::WorkPackagesByWorkspaceAPI
mount API::V3::Categories::CategoriesByWorkspaceAPI
mount API::V3::Versions::VersionsByProjectAPI
mount API::V3::Labels::LabelsByWorkspaceAPI
mount API::V3::Queries::QueriesByWorkspaceAPI
mount API::V3::Favorites::FavoriteActionsAPI, with: { favorite_object_getter: ->(*) { @project } }
end
Expand Down
6 changes: 6 additions & 0 deletions spec/lib/api/v3/utilities/path_helper_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,12 @@
it_behaves_like "api v3 path", "/versions/available_projects"
end

describe "#labels_by_workspace" do
subject { helper.labels_by_workspace 42 }

it_behaves_like "api v3 path", "/workspaces/42/labels"
end

describe "#versions_by_project" do
subject { helper.versions_by_project 42 }

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -231,5 +231,61 @@
expect(new_checksum)
.not_to eql orig_checksum
end

it "produces a different checksum when a label is added to the work package" do
label = create(:label)

create(:labeling, labelable: work_package, label:)

expect(new_checksum)
.not_to eql orig_checksum
end

it "produces a different checksum when a label is removed from the work package" do
label = create(:label)
labeling = create(:labeling, labelable: work_package, label:)

previous_checksum = EagerLoadingMockWrapper
.wrap(described_class, [work_package])
.first
.cache_checksum

labeling.destroy!

expect(new_checksum)
.not_to eql previous_checksum
end

it "produces a different checksum when an assigned label is renamed" do
label = create(:label)
create(:labeling, labelable: work_package, label:)

previous_checksum = EagerLoadingMockWrapper
.wrap(described_class, [work_package])
.first
.cache_checksum

label.update_attribute(:name, "Renamed label #{label.id}")

expect(new_checksum)
.not_to eql previous_checksum
end

it "produces the same checksum when a label not assigned to the work package changes" do
assigned_label = create(:label)
create(:labeling, labelable: work_package, label: assigned_label)
unassigned_label = create(:label)
create(:labeling, labelable: create(:work_package, project:), label: unassigned_label)

previous_checksum = EagerLoadingMockWrapper
.wrap(described_class, [work_package])
.first
.cache_checksum

unassigned_label.update_attribute(:name, "Renamed label #{unassigned_label.id}")

expect(new_checksum)
.to eql previous_checksum
end
end
end
Loading
Loading