74233 Commits

Author SHA1 Message Date
Fredrik Adelöw f1d243293b test(catalog-backend): await engine start/stop in integration test harness
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-22 11:42:42 +02:00
Fredrik Adelöw 0e1c7e9504 Expose totalItemsLoading and improve table loading UX
Add totalItemsLoading to EntityListContextProps so consumers can
distinguish a stale count (being refreshed) from a fresh one. Rename
the internal asyncTotalItems to totalItems.

Update CatalogTable to show a dimmed count while the count is loading,
and only show the full-table spinner on the very first load (when no
entities exist yet). Subsequent fetches keep stale rows visible with
a small spinner next to the title.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-22 10:57:03 +02:00
Aramis Sennyey ac410b1d26 feat(skill): automatically onboard to openapi tooling (#34179)
* feat(skill): automatically onboard to openapi tooling

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* test skill against notifications backend

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* update URLs to point to main repo and describe multiple client types

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* address PR feedback

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* revert erroneous changes

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* add changeset

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* address review feedback: skill doc fixes and entityRef schema dedup

Disambiguate generated entityRef/orderField types and correct skill
guidance on operationId casing and OpenAPI version support.

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

* note in skill that handler-side validation should be stripped

Reviewers may forget that manual InputError guards and primitive-type
checks become dead weight once the OpenAPI validator runs.

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>

---------

Signed-off-by: aramissennyeydd <aramis.sennyey@doordash.com>
2026-05-21 23:01:16 -04:00
Fredrik Adelöw eac181f1b5 fix(catalog-backend): fix immediate mode logging on repeated calls
A second call with mode='immediate' would incorrectly log "Unknown
strategy mode" because the deprecation flag was already set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 22:38:04 +02:00
dependabot[bot] e23f05c33a chore(deps): bump @tootallnate/once from 2.0.0 to 2.0.1 in /microsite
Bumps [@tootallnate/once](https://github.com/TooTallNate/once) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/TooTallNate/once/releases)
- [Changelog](https://github.com/TooTallNate/once/blob/v2.0.1/CHANGELOG.md)
- [Commits](https://github.com/TooTallNate/once/compare/2.0.0...v2.0.1)

---
updated-dependencies:
- dependency-name: "@tootallnate/once"
  dependency-version: 2.0.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-21 17:52:42 +00:00
Patrik Oldsberg a7d7542432 Merge pull request #31837 from the-serious-programmer/feature/catalog-export
Feature: catalog export
2026-05-21 19:50:30 +02:00
dependabot[bot] b23bc03183 chore(deps): bump @tootallnate/once from 1.1.2 to 2.0.1
Bumps [@tootallnate/once](https://github.com/TooTallNate/once) from 1.1.2 to 2.0.1.
- [Release notes](https://github.com/TooTallNate/once/releases)
- [Changelog](https://github.com/TooTallNate/once/blob/v2.0.1/CHANGELOG.md)
- [Commits](https://github.com/TooTallNate/once/compare/1.1.2...v2.0.1)

---
updated-dependencies:
- dependency-name: "@tootallnate/once"
  dependency-version: 2.0.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-21 17:11:33 +00:00
Fredrik Adelöw e335055c40 fix(catalog-backend): use hash check for stitch ticket guard instead of unreliable row counts
INSERT return values differ across database engines (row IDs, row
counts, or empty arrays), making them unsuitable for detecting blocked
upserts. Use a post-write hash verification instead, which works
reliably on all engines.

Adds test coverage for stale ticket rejection across all supported
database engines (MySQL 8, PostgreSQL 14/18, SQLite 3).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 18:07:12 +02:00
Fredrik Adelöw 9772b28bd8 Restore dedup test and add split-count coverage
Restore the 'does not re-fetch when backend filter params are
unchanged' test from #34324 that was accidentally dropped during
the rebase. Add a new test verifying that cursor navigation does
not re-run the count query (the core split-count behavior).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 18:04:40 +02:00
Fredrik Adelöw 8d38538a82 fix(catalog-backend): fix tsc error — Knex insert returns number[], not number
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 18:02:09 +02:00
Fredrik Adelöw e864f77e35 fix(catalog-backend): make stitch ticket guard atomic on PostgreSQL and SQLite
On PostgreSQL and SQLite, the stitch ticket check is now part of the
upsert WHERE clause, eliminating the TOCTOU window between the check
and the write. MySQL does not support ON CONFLICT ... DO UPDATE ... WHERE,
so it retains the separate SELECT with a negligible race window.

SQLite misreports affected rows for blocked upserts, so a post-write
hash verification is used there instead of the row count.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:31 +02:00
Fredrik Adelöw 5e6af9b7f6 fix(catalog-backend): fix stitcher error message and warn on unknown strategy mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:31 +02:00
Fredrik Adelöw b8731fc82b Remove unused DbRefreshStateRow import and logger declaration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:31 +02:00
Fredrik Adelöw 7bab21f1ef Remove redundant entity existence check in performStitching
The initial SELECT entity_id FROM refresh_state check is redundant —
the processedResult query in the Promise.all already handles the
missing-entity case. This saves one database round trip per stitch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:31 +02:00
Fredrik Adelöw bcbc92d2ad Bump changeset to minor for config schema change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw 4282877ba9 Restore log-once guard for immediate mode deprecation warning
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw ed2a9fe492 Replace placeholder final_entities insert with upsert
Now that immediate mode is removed, the two-step pattern of inserting a
placeholder row and then updating it is unnecessary. Replace with a
single upsert that creates or updates the final_entities row in one
operation, removing a round-trip and simplifying the flow.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw 640f6b37a3 Fix literal control chars in scriptProtocolPattern regex
The conflict resolution replaced unicode escapes with literal control
character bytes, causing GitHub to render the file as binary.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw d0d1b9b5b9 Run prettier on files with formatting issues
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw dc7678cdcc fix(catalog-backend): remove immediate mode stitching
Remove the immediate stitching strategy from the catalog-backend plugin.
All stitching now uses the deferred mode exclusively, which processes
entities asynchronously via a worker queue. This simplifies the stitching
subsystem by removing dead-weight code paths and the strategy parameter
threading, and fixes a bug where deleteWithEagerPruningOfChildren
hardcoded immediate-mode semantics regardless of configured mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:59:30 +02:00
Fredrik Adelöw d8757b158c catalog-react: split entity list and count into parallel requests
Reapply the split-count concept on top of the refactored
useEntityListProvider (from #34324). The list fetch now uses
totalItems: 'exclude' to skip the expensive count, and a separate
useAsyncFn fetches the count only when filters change — page
navigation no longer re-runs the count query.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 17:02:49 +02:00
Fredrik Adelöw 2139bf53d1 Set cursor totalItemsMode to exclude with explanatory comment
Cursor requests already carry the computed totalItems from page 1,
so the count query is skipped regardless of this value. Setting it
to 'exclude' reflects what actually happens, and the comment
explains why the value doesn't matter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 16:52:47 +02:00
Fredrik Adelöw 4cc7231379 Update changeset to reflect delta on top of #34323
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 16:52:47 +02:00
Fredrik Adelöw 8f20cc2f52 Add totalItems mode, sort key lowercasing, and simplified ordering
Layer the remaining concepts from the original PR on top of the
split-query foundation:

- totalItems parameter on REST API (GET and POST) and CatalogApi
- TotalItemsMode type replacing internal skipTotalItems boolean
- Sort field key lowercasing for search table comparison
- whereNotNull(search.value) on the list CTE to exclude truncated values
- Simplified ORDER BY (NULLS LAST removed since NULL values are
  already excluded by whereNotNull)
- New tests for NULL sort field values and multi-valued sort field
  totalItems accuracy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 16:52:47 +02:00
Fredrik Adelöw 74c5fbbafd Merge pull request #34324 from backstage/freben/fix-entity-list-triple-fetch
fix(catalog-react): eliminate redundant entity list fetches during filter initialization
2026-05-21 16:48:32 +02:00
Fredrik Adelöw 5dbbb15574 Merge pull request #34323 from backstage/freben/split-query-entities-count
fix(catalog-backend): split queryEntities list and count queries
2026-05-21 15:56:09 +02:00
Fredrik Adelöw 35c473d736 Merge pull request #34273 from backstage/freben/catalog-db-perf-skill
catalog-backend: add query performance battery and baseline
2026-05-21 15:55:30 +02:00
Fredrik Adelöw fcde31d871 Address review: remove fallback route, handle trailing splat, extract shared path
- Remove the path="*" fallback Route so mismatches between mountPath
  and initialRouteEntries surface as test failures instead of silently
  rendering without params
- Handle mountPath values that already end with /* to avoid double splat
- Extract repeated entity path string into a shared constant

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 15:49:36 +02:00
Fredrik Adelöw bc1b4263c9 Update API report for frontend-test-utils
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 15:40:39 +02:00
Fredrik Adelöw 5dca781385 Replace mountedRoutes-based wrapping with mountPath option
Address review feedback: mountedRoutes are for abstract route targets
the test subject links to, not for the subject itself. Add a dedicated
mountPath option that wraps the test element in a single Route with
the given path pattern, enabling useParams(). When mountPath is set
and initialRouteEntries is not, the initial entry defaults to mountPath.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 13:39:52 +02:00
Vincenzo Scamporlino 80b4370413 Add instructions for enabling the catalog-unprocessed-entities page
Signed-off-by: Vincenzo Scamporlino <vincenzos@spotify.com>
2026-05-21 12:48:15 +02:00
Vincenzo Scamporlino bc78e770c0 Update README to clarify plugin activation and DevTools integration
Signed-off-by: Vincenzo Scamporlino <vincenzos@spotify.com>
2026-05-21 12:48:01 +02:00
Fredrik Adelöw 392c1b2851 refactor(catalog-react): drop useCallback on refresh — useDebounce uses a ref internally
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 10:18:24 +02:00
Fredrik Adelöw 77e7ad4c7d fix(catalog-react): clear fetch dedup ref on failure instead of reverting to previous params
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 10:12:51 +02:00
Fredrik Adelöw 720fb542d9 cleanup(catalog-react): remove dead appliedCursor field, fix cursor-mode guard, inline getPaginationMode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-21 09:55:57 +02:00
Fredrik Adelöw 9e880b70fd Deduplicate adjustedFilters computation
Move the adjustedFilters useMemo above refresh and reuse it in both
the fetch callback and the frontend filtering memo.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 22:42:49 +02:00
renovate[bot] 6b58111494 Update ghcr.io/devcontainers/features/common-utils Docker tag to v2.5.9
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-20 20:27:38 +00:00
1337 9929df3d8c Merge branch 'backstage:master' into feature/catalog-export 2026-05-20 20:32:27 +02:00
Fredrik Adelöw ffa59be219 Revert lastFetchParamsRef on failed fetch to allow retries
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 20:24:34 +02:00
Fredrik Adelöw 779749fbd7 Guard against empty-filter fetch on mount and premature URL sync
The lastFetchParamsRef starts as undefined, so the initial debounce
would have fetched with an empty filter (returning every entity in
the catalog). Guard against this by skipping the fetch when no filter
components have registered yet.

Also skip the URL sync effect until at least one filter has been set,
to avoid briefly clearing filter query params from the URL bar before
filter components initialize.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:57:33 +02:00
Fredrik Adelöw 4694eb16e3 Revert EntityKindPicker ref — provider dedup handles it
The label is read by CatalogTable for the table title (e.g. "All
Components (42)"), so suppressing the label update would cause a
visible regression. The provider's ref-based fetch dedup already
prevents the redundant API call when only the label changes, so
the second updateFilters call is harmless.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:37:35 +02:00
Fredrik Adelöw 7c205457b6 Add changeset for entity list triple-fetch fix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:31:38 +02:00
Fredrik Adelöw a103f0e604 test(catalog-react): add regression test for entity list fetch dedup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:30:49 +02:00
Fredrik Adelöw 4222c73fd3 fix(catalog-react): eliminate redundant fetches in useEntityListProvider
Replace useAsyncFn + outputState comparison with ref-based fetch dedup
and generation counter. Split entity filtering into a synchronous
useMemo so frontend-only filter changes are instant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:29:44 +02:00
Fredrik Adelöw dfb920fb6e fix(catalog-react): prevent EntityKindPicker from double-firing updateFilters on label change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 17:28:00 +02:00
Fredrik Adelöw 2c4b4912c2 Add whereNotNull to count query sort-field EXISTS
The list CTE's INNER JOIN on search already excludes entities whose
sort-field value is NULL (truncated long values stored as NULL by
buildEntitySearch). Add the same whereNotNull to the count query's
EXISTS so totalItems matches the reachable set. Matters for fields
like metadata.description where ~128K entities have NULL values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 16:59:39 +02:00
Fredrik Adelöw 02a6410992 Extract shared applyPredicates helper to deduplicate filter logic
The list CTE and count query both need the same filter, query, and
full-text-search predicates. Extract them into a shared
applyPredicates closure so they can't drift apart. Move the count
query construction next to the CTE for readability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 16:35:41 +02:00
Fredrik Adelöw 59fef3e80b Add patch entry for #34323, remove shipped patches
Remove pr-33721, pr-34001, pr-34004 — all shipped in v1.51.0.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 16:31:30 +02:00
Fredrik Adelöw 0119806204 Update changeset to document totalItems semantic fix
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Fredrik Adelöw <freben@gmail.com>
2026-05-20 16:29:17 +02:00
Andre Wanlin 097ab7e98f Contributing - Review Process Updates
Signed-off-by: Andre Wanlin <awanlin@spotify.com>

Fixed typo

Signed-off-by: Andre Wanlin <awanlin@spotify.com>

Fixed another typo

Signed-off-by: Andre Wanlin <awanlin@spotify.com>

Addressed comments

Signed-off-by: Andre Wanlin <awanlin@spotify.com>
2026-05-20 08:56:59 -05:00