diff --git a/docs/backend-system/building-backends/08-migrating.md b/docs/backend-system/building-backends/08-migrating.md index 98dde46d38..b20e5a98e1 100644 --- a/docs/backend-system/building-backends/08-migrating.md +++ b/docs/backend-system/building-backends/08-migrating.md @@ -262,9 +262,259 @@ If you have other customizations made to `plugins/catalog.ts`, such as adding custom processors or entity providers, read on. Otherwise, you should be able to just delete that file at this point. +#### Amazon Web Services + +`AwsEksClusterProcessor` and `AwsOrganizationCloudAccountProcessor` have not yet been migrated to the new backend system. +See [Other Catalog Extensions](#other-catalog-extensions) for how to use these in the new backend system. + +For `AwsS3DiscoveryProcessor`, first migrate to `AwsS3EntityProvider`. + +To migrate `AwsS3EntityProvider` to the new backend system, add a reference to the `@backstage/plugin-catalog-backend-module-aws` module. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-aws/alpha')); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other AWS configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + awsS3: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT1H + timeout: PT50M + /* highlight-add-end */ +``` + +#### Azure Devops + +For `AzureDevOpsDiscoveryProcessor`, first migrate to `AzureDevOpsEntityProvider`. + +To migrate `AzureDevOpsEntityProvider` to the new backend system, add a reference to the `@backstage/plugin-catalog-backend-module-azure` module. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-azure/alpha')); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Azure DevOps configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + azureDevOps: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT1H + timeout: PT50M + /* highlight-add-end */ +``` + +#### Open API + +`InternalOpenApiDocumentationProvider` have not yet been migrated to the new backend system. +See [Other Catalog Extensions](#other-catalog-extensions) for how to use this in the new backend system. + +#### Bitbucket + +For `BitbucketDiscoveryProcessor`, migrate to `BitbucketCloudEntityProvider` or `BitbucketServerEntityProvider` + +To migrate `BitbucketCloudEntityProvider` to the new backend system, add a reference to the `@backstage/plugin-catalog-backend-module-bitbucket-cloud` module. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add( + import('@backstage/plugin-catalog-backend-module-bitbucket-cloud/alpha'), +); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Bitbucket Cloud configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + bitbucketCloud: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT30M + timeout: PT3M + /* highlight-add-end */ +``` + +To migrate `BitbucketServerEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-bitbucket-server`. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add( + import('@backstage/plugin-catalog-backend-module-bitbucket-server/alpha'), +); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Bitbucket Server configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + bitbucketServer: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT30M + timeout: PT3M + /* highlight-add-end */ +``` + +#### Google Cloud Platform + +To migrate `GkeEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-gcp`. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-gcp')); +/* highlight-add-end */ +``` + +Configuration in app-config.yaml remains the same. + +#### Gerrit + +To migrate `GerritEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-gerrit`. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-gerrit/alpha')); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Gerrit configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + gerrit: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT30M + timeout: PT3M + /* highlight-add-end */ +``` + +#### Github + +For `GithubDiscoveryProcessor`, `GithubMultiOrgReaderProcessor` and `GithubOrgReaderProcessor`, first migrate to the equivalent Entity Provider. + +To migrate `GithubEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-github`. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-github/alpha')); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Github configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + github: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT30M + timeout: PT3M + /* highlight-add-end */ +``` + +To migrate `GithubMultiOrgEntityProvider` and `GithubOrgEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-github-org`. + +```ts title="packages/backend/src/index.ts" +backend.add(import('@backstage/plugin-catalog-backend/alpha')); +/* highlight-add-start */ +backend.add(import('@backstage/plugin-catalog-backend-module-github-org')); +/* highlight-add-end */ +``` + +If you were providing a `schedule` in code, this now needs to be set via configuration. +All other Github configuration in `app-config.yaml` remains the same. + +```yaml title="app-config.yaml" +catalog: + providers: + githubOrg: + yourProviderId: + # ... + /* highlight-add-start */ + schedule: + frequency: PT30M + timeout: PT3M + /* highlight-add-end */ +``` + +If you were providing transformers, these can be configured by extending `githubOrgEntityProviderTransformsExtensionPoint` + +```ts title="packages/backend/src/index.ts" +import { createBackendModule } from '@backstage/backend-plugin-api'; +import { githubOrgEntityProviderTransformsExtensionPoint } from '@backstage/plugin-catalog-backend-module-github-org'; + +backend.add( + createBackendModule({ + pluginId: 'catalog', + moduleId: 'githubOrgTransformers', + register(env) { + env.registerInit({ + deps: { + /* highlight-add-start */ + githubOrgTransformers: + githubOrgEntityProviderTransformsExtensionPoint, + /* highlight-add-end */ + }, + async init({ githubOrgTransformers }) { + /* highlight-add-start */ + githubOrgTransformers.setUserTransformer(myUserTransformer); + githubOrgTransformers.setTeamTransformer(myTeamTransformer); + /* highlight-add-end */ + }, + }); + }, + }), +); +``` + #### Microsoft Graph -Import the Microsoft Graph catalog module +For `MicrosoftGraphOrgReaderProcessor`, first migrate to `MicrosoftGraphOrgEntityProvider` + +To migrate `MicrosoftGraphOrgEntityProvider` to the new backend system, add a reference to `@backstage/plugin-catalog-backend-module-github-org`. ```ts title="packages/backend/src/index.ts" backend.add(import('@backstage/plugin-catalog-backend/alpha')); @@ -273,7 +523,8 @@ backend.add(import('@backstage/plugin-catalog-backend-module-msgraph/alpha')); /* highlight-add-end */ ``` -If you were providng a `schedule` programtically, this now needs to be set via configuration +If you were providing a `schedule` in code, this now needs to be set via configuration +All other Microsoft Graph configuration in `app-config.yaml` remains the same. ```yaml title="app-config.yaml" catalog: @@ -285,7 +536,6 @@ catalog: frequency: PT4H timeout: PT30M /* highlight-add-end */ - ``` If you were providing transformers, these can be configured by extending `microsoftGraphOrgEntityProviderTransformExtensionPoint` @@ -297,7 +547,7 @@ import { microsoftGraphOrgEntityProviderTransformExtensionPoint } from '@backsta backend.add( createBackendModule({ pluginId: 'catalog', - moduleId: 'microsoftGraphExtensions', + moduleId: 'microsoftGraphTransformers', register(env) { env.registerInit({ deps: { diff --git a/docs/integrations/aws-s3/discovery.md b/docs/integrations/aws-s3/discovery.md index 4a2b953bc9..ceb780141f 100644 --- a/docs/integrations/aws-s3/discovery.md +++ b/docs/integrations/aws-s3/discovery.md @@ -32,7 +32,7 @@ catalog: bucketName: sample-bucket prefix: prefix/ # optional region: us-east-2 # optional, uses the default region otherwise - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # same options as in TaskScheduleDefinition # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code @@ -52,7 +52,7 @@ catalog: bucketName: sample-bucket prefix: prefix/ # optional region: us-east-2 # optional, uses the default region otherwise - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # same options as in TaskScheduleDefinition # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code @@ -79,12 +79,6 @@ const builder = await CatalogBuilder.create(env); builder.addEntityProvider( AwsS3EntityProvider.fromConfig(env.config, { logger: env.logger, - // optional: alternatively, use scheduler with schedule defined in app-config.yaml - schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }), - // optional: alternatively, use schedule scheduler: env.scheduler, }), ); diff --git a/docs/integrations/azure/discovery.md b/docs/integrations/azure/discovery.md index 7b91308b56..3b25a9ac75 100644 --- a/docs/integrations/azure/discovery.md +++ b/docs/integrations/azure/discovery.md @@ -71,7 +71,7 @@ The parameters available are: - **`repository:`** _(optional)_ The repository name. Wildcards are supported as show on the examples above. If not set, all repositories will be searched. - **`path:`** _(optional)_ Where to find catalog-info.yaml files. Defaults to /catalog-info.yaml. - **`branch:`** _(optional)_ The branch name to use. -- **`schedule`** _(optional)_: +- **`schedule`**: - **`frequency`**: How often you want the task to run. The system does its best to avoid overlapping invocations. - **`timeout`**: diff --git a/docs/integrations/bitbucketCloud/discovery.md b/docs/integrations/bitbucketCloud/discovery.md index 6e1533dea0..53315a4c48 100644 --- a/docs/integrations/bitbucketCloud/discovery.md +++ b/docs/integrations/bitbucketCloud/discovery.md @@ -49,19 +49,6 @@ export default async function createPlugin( } ``` -Alternatively to the config-based schedule, you can use - -```ts -/* highlight-remove-next-line */ -scheduler: env.scheduler, -/* highlight-add-start */ -schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, -}), -/* highlight-add-end */ -``` - ### Installation with Events Support Please follow the installation instructions at @@ -131,7 +118,7 @@ catalog: filters: # optional projectKey: '^apis-.*$' # optional; RegExp repoSlug: '^service-.*$' # optional; RegExp - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # same options as in TaskScheduleDefinition # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code @@ -153,7 +140,7 @@ catalog: Regular expression used to filter results based on the project key. - **`repoSlug`** _(optional)_: Regular expression used to filter results based on the repo slug. -- **`schedule`** _(optional)_: +- **`schedule`**: - **`frequency`**: How often you want the task to run. The system does its best to avoid overlapping invocations. - **`timeout`**: diff --git a/docs/integrations/bitbucketServer/discovery.md b/docs/integrations/bitbucketServer/discovery.md index 05d0510553..ad59006ff1 100644 --- a/docs/integrations/bitbucketServer/discovery.md +++ b/docs/integrations/bitbucketServer/discovery.md @@ -38,12 +38,6 @@ export default async function createPlugin( builder.addEntityProvider( BitbucketServerEntityProvider.fromConfig(env.config, { logger: env.logger, - // optional: alternatively, use scheduler with schedule defined in app-config.yaml - schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }), - // optional: alternatively, use schedule scheduler: env.scheduler, }), ); @@ -69,7 +63,7 @@ catalog: filters: # optional projectKey: '^apis-.*$' # optional; RegExp repoSlug: '^service-.*$' # optional; RegExp - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # same options as in TaskScheduleDefinition # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code @@ -87,7 +81,7 @@ catalog: Regular expression used to filter results based on the project key. - **`repoSlug`** _(optional)_: Regular expression used to filter results based on the repo slug. -- **`schedule`** _(optional)_: +- **`schedule`**: - **`frequency`**: How often you want the task to run. The system does its best to avoid overlapping invocations. - **`timeout`**: diff --git a/docs/integrations/gerrit/discovery.md b/docs/integrations/gerrit/discovery.md index 5d822effb5..ee843d2454 100644 --- a/docs/integrations/gerrit/discovery.md +++ b/docs/integrations/gerrit/discovery.md @@ -26,18 +26,11 @@ Then add the plugin to the plugin catalog `packages/backend/src/plugins/catalog. ```ts /* packages/backend/src/plugins/catalog.ts */ import { GerritEntityProvider } from '@backstage/plugin-catalog-backend-module-gerrit'; -import { Duration } from 'luxon'; const builder = await CatalogBuilder.create(env); /** ... other processors and/or providers ... */ builder.addEntityProvider( GerritEntityProvider.fromConfig(env.config, { logger: env.logger, - // optional: alternatively, use scheduler with schedule defined in app-config.yaml - schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }), - // optional: alternatively, use schedule scheduler: env.scheduler, }), ); @@ -57,7 +50,7 @@ catalog: host: gerrit-your-company.com branch: master # Optional query: 'state=ACTIVE&prefix=webapps' - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code diff --git a/docs/integrations/github/discovery.md b/docs/integrations/github/discovery.md index 46d59cbf1b..3c3d90d6e9 100644 --- a/docs/integrations/github/discovery.md +++ b/docs/integrations/github/discovery.md @@ -40,12 +40,6 @@ export default async function createPlugin( builder.addEntityProvider( GithubEntityProvider.fromConfig(env.config, { logger: env.logger, - // optional: alternatively, use scheduler with schedule defined in app-config.yaml - schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }), - // optional: alternatively, use schedule scheduler: env.scheduler, }), ); @@ -85,12 +79,6 @@ export default async function createPlugin( /* highlight-add-start */ const githubProvider = GithubEntityProvider.fromConfig(env.config, { logger: env.logger, - // optional: alternatively, use scheduler with schedule defined in app-config.yaml - schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 30 }, - timeout: { minutes: 3 }, - }), - // optional: alternatively, use schedule scheduler: env.scheduler, }); env.eventBroker.subscribe(githubProvider); @@ -122,7 +110,7 @@ catalog: filters: branch: 'main' # string repository: '.*' # Regex - schedule: # optional; same options as in TaskScheduleDefinition + schedule: # same options as in TaskScheduleDefinition # supports cron, ISO duration, "human duration" as used in code frequency: { minutes: 30 } # supports ISO duration, "human duration" as used in code @@ -213,7 +201,7 @@ This provider supports multiple organizations via unique provider IDs. Defaults to `false`. Due to limitations in the GitHub API's ability to query for repository objects, this option cannot be used in conjunction with wildcards in the `catalogPath`. -- **`schedule`** _(optional)_: +- **`schedule`**: - **`frequency`**: How often you want the task to run. The system does its best to avoid overlapping invocations. - **`timeout`**: @@ -229,13 +217,12 @@ GitHub [rate limits](https://docs.github.com/en/rest/overview/resources-in-the-r accounts). The snippet below refreshes the Backstage catalog data every 35 minutes, which issues an API request for each discovered location. If your requests are too frequent then you may get throttled by -rate limiting. You can change the refresh frequency of the catalog in your `packages/backend/src/plugins/catalog.ts` file: +rate limiting. You can change the refresh frequency of the catalog in your `app-config.yaml` file by controlling the `schedule`. -```typescript -schedule: env.scheduler.createScheduledTaskRunner({ - frequency: { minutes: 35 }, - timeout: { minutes: 30 }, -}), +```yaml +schedule: + frequency: { minutes: 35 } + timeout: { minutes: 3 } ``` More information about scheduling can be found on the [TaskScheduleDefinition](https://backstage.io/docs/reference/backend-tasks.taskscheduledefinition) page.