Commit Graph

62 Commits

Author SHA1 Message Date
Himanshu Mishra f454b9f865 TechDocs/AWS: Update tutorial with auth best practices 2021-01-12 12:35:13 +01:00
Himanshu Mishra 16b73697a3 TechDocs/AWS: Enable authentication using env variables and ~/.aws/config shared file
If authentication secrets are provided in app-config.yaml, it will be used. If not, environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION will be used. If not present, ~/.aws/config will be used to read the configs.

https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-environment.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html
2021-01-12 11:41:00 +01:00
Fredrik Adelöw 1c7771871e Merge pull request #3953 from ayshiff/feature/techdocs-aws-sdkv3
TechDocs: AWS S3 SDK version bump
2021-01-12 10:21:51 +01:00
Himanshu Mishra 3790b585c9 Merge pull request #3966 from backstage/orkohunter/techdocs-cli-uses-techdocs-common 2021-01-11 23:23:26 +01:00
Himanshu Mishra 078987e251 techdocs-common: fix tests for the new generator API 2021-01-11 23:08:30 +01:00
Himanshu Mishra 2165901c00 TechDocs: Generator now requires input and output directory and does not create them
The creation of temporary directories and their clean up should be handled on its own. techdocs-cli already has a fixed input output dir requirement. It makes that generator accepts these values as requirements.
2021-01-11 22:33:10 +01:00
Himanshu Mishra 15cc6ff12c Merge pull request #3981 from backstage/orkohunter/improve-gcs-credentials-handling 2021-01-11 21:12:55 +01:00
Himanshu Mishra 97716f2b04 Apply suggestions from code review
Co-authored-by: Fredrik Adelöw <freben@gmail.com>
2021-01-11 20:50:20 +01:00
Remi 5826d0973e feat(techdocs): add changeset + refactor 2021-01-11 18:19:28 +01:00
Remi b18efa0553 feat(techdocs): streamToString try/catch 2021-01-11 17:21:50 +01:00
Remi 60cad4c1ab fix(techdocs): getObject stream 2021-01-11 17:21:50 +01:00
Remi 33b1a43c77 fix(techdocs): broken windows tests 2021-01-11 17:21:50 +01:00
Remi 7f34d22bdf feat(techdocs): update aws-sdk to v3 2021-01-11 17:21:50 +01:00
Himanshu Mishra de4c3af867 fix(techdocs): Handle images and other binaries correctly in aws/gcs publishers
Closes https://github.com/backstage/backstage/issues/3990
2021-01-10 10:09:42 +01:00
Himanshu Mishra e7954559ab TechDocs/GCS: Remove projectId since it is not needed
I initially thought the GCS Nodejs client would make use of it, since it asked for it in the contructor. However, we may not need them at all. Buckets are gloablly unique anyway.
2021-01-09 16:29:42 +01:00
Himanshu Mishra 1ef5d0f13c TechDocs/GCS: Enable authentication using environemnt variables
GOOGLE_APPLICATION_CREDENTIALS is the environment variable used by the client. Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions automatically provide a default service account and set this env variable.
https://cloud.google.com/docs/authentication/production
2021-01-09 16:07:39 +01:00
Himanshu Mishra c65dc26c10 TechDocs: Generator can be given a pre-created output directory
TechDocs Generator uses a temporary directory to store the generated files. But with techdocs-cli, the generator needs to output in a fixed directory.

Also, make parsedLocationAnnotation optional since it is not a critical requirement for the generator.
2021-01-08 13:42:55 +01:00
Remi a87fc11aca tests(techdocs): refactor 2021-01-05 21:31:18 +01:00
Remi 0f40f1934f fix(techdocs): path delimiters 2021-01-05 18:35:23 +01:00
Himanshu Mishra 0b9646ed6a Merge pull request #3794 from ayshiff/feature/techdocs-aws-s3 2021-01-05 17:48:57 +01:00
Ben Lambert cf2c2df7d3 Merge pull request #3834 from erikxiv/fix/techdocs-ghe-apibaseurl
Fix for integration.github.apiBaseUrl configuration not properly overriding apiBaseUrl used by techdocs
2021-01-04 13:18:22 +01:00
blam 5b38fd70d6 chore: it's safe by default now, so we can remove the nesting 2021-01-04 11:39:22 +01:00
blam 9ca93c9f7a chore: udpate usage in techdocs-common too to avoid pulling in two versions of js-yam 2021-01-04 11:22:15 +01:00
Remi 1517009d8a refactor(techdocs-common): add try/catch statements + fs-extra 2021-01-03 03:40:21 +01:00
Himanshu Mishra dc99b07447 fix(techdocs-common) broken test
also copy yarn.lock from master; and yarn again

fix(techdocs-common): broken async test
2020-12-30 10:25:58 +01:00
Himanshu Mishra e4dfaba3c1 techdocs: fix metadata API response from S3 storage 2020-12-30 10:25:57 +01:00
Remi 80b60862af feat(techdocs): updates aws variables access 2020-12-30 10:25:57 +01:00
Remi 0661c4f725 fix(techdocs-common): format 2020-12-30 10:25:57 +01:00
Remi b9e6d305e2 fix(techdocs): typos + add tests + requested changes 2020-12-30 10:25:57 +01:00
Remi 134888fb21 fix(techdocs): format + doc quality check 2020-12-30 10:25:57 +01:00
Remi 772c7fca4b feat(techdocs-common): add core awsS3 2020-12-30 10:22:12 +01:00
Remi 2d0ab25779 feat(techdocs-common): add awsS3 tests 2020-12-30 10:22:12 +01:00
Remi 6af0abb403 feat(techdocs-common): add awsS3 2020-12-30 10:22:12 +01:00
blam 8bccc6bec0 chore: refactor variable names for isomorphic git. 2020-12-29 13:32:24 +01:00
Himanshu Mishra 231b55b874 Merge remote-tracking branch 'origin/master' into blam/isomorphic-git 2020-12-28 22:02:46 +01:00
Himanshu Mishra 8804e89811 techdocs: Add changeset for @backstage/integration changes 2020-12-28 21:18:18 +01:00
Himanshu Mishra c91e4d6b04 techdocs: Use @backstage/integration for scm tokens and request options 2020-12-28 21:08:54 +01:00
blam f60b02bec4 Merge branch 'master' of github.com:backstage/backstage into blam/isomorphic-git
* 'master' of github.com:backstage/backstage: (23 commits)
  cli: fix windows tests
  Introduce slight bias towards positive trend in cost insights mock data
  core-api: rename BackstageRoutes to FlatRoutes
  Update plugins/catalog/src/hooks/useEntity.ts
  Fix parameter name in docstring for duration#inclusiveStartDateOf
  Adjust test
  Add changeset
  fix: token expiration in s, not ms
  Create loud-days-breathe.md
  Create polite-glasses-occur.md
  dev-utils: add a small label with the entity name to EntityGridItems
  dev-utils: add EntityGridItem for easily displaying various entity cards
  dev-utils: use dev index module as root for hot reloading
  dev-utils: add .registerPage for use with extensions
  catalog: rename EntityProvider to EntityLoaderProvider, and add new EntityProvider
  reduce close timeout to 100 ms
  changeset: add changeset for core-api RouteRef deprecations
  core-api: use typescript workaround for using private constructor instead of a runtime one
  core-api: deprecate RouteRef path and remove deprecated createSubRoute
  delay window close by 200 ms
  ...
2020-12-28 17:39:15 +01:00
Fredrik Adelöw 0574a27b7c cli: fix windows tests 2020-12-28 11:43:25 +01:00
blam cc4a49aaf3 Merge branch 'master' of github.com:backstage/backstage into blam/isomorphic-git
* 'master' of github.com:backstage/backstage: (45 commits)
  Speed up some PRs with paths-ignore
  Remove unnecessary step
  catalog: export EntitySwitch component and conditions
  Apply suggestions from code review
  changeset: add changeset for new catalog composability components
  catalog: fix EntityLayout TabbedLayout route path matching
  catalog: gather all mount points beneath EntityLayout Routes
  catalog: add EntityLayout
  Change to raw Windows Discord hook
  Release plugin-cost-insights 0.5.3
  catalog: add EntitySwitch
  catalog: export new CatalogIndexPage and CatalogEntityPage
  Create dull-seals-march.md
  core-api: add BackstageRoutes
  Fix plugin name
  fixed category lables
  feat(cost-insights): Make alert url field optional
  fossa: Add changeset
  fix(fossa): Bump core packages version to fix failing CI
  Update 2020-12-22-stability-index.md
  ...
2020-12-26 22:22:11 +01:00
Himanshu Mishra cb28e1a6a0 chore(git): Need author and commiter for doing git merge 2020-12-23 00:34:12 +01:00
Himanshu Mishra 9ca586e579 techdocs: Replace nodegit with isomorphic-git
TechDocs needs 4 functions from the git library
1. To clone 2. To get the commit timestamp for HEAD 3. To fetch 4. To merge two branches

Created necessary commands in the new SCM client and reordered them alphabetically for convenience
2020-12-23 00:12:20 +01:00
Erik Larsson 186604d350 Fix for 'integration.github.apiBaseUrl' configuration not properly overriding apiBaseUrl used by techdocs. 2020-12-22 23:10:19 +01:00
Himanshu Mishra 2ac77aeec2 techdocs-backend: Remove individual preparers and generators from app backend, to use factory methods
When a new Backstage app is setup, the techdocs backend file contains all the preparers (dir, github, gitlab, etc.) and same for generators. While this providers added customizibility, it comes with a cost of complexity in setting up the app backend. New preparers' updates would need users to update their app's techdocs backend file.

Scaffolder backend has already moved to this proposed pattern.
2020-12-17 11:33:56 +01:00
Himanshu Mishra d62be2f822 Merge pull request #3731 from backstage/orkohunter/techdocs-set-correct-content-type-header 2020-12-16 10:58:04 +01:00
Himanshu Mishra ef4927baf6 techdocs-common: Use || operator instead of full if-else checks 2020-12-15 22:19:03 +01:00
Himanshu Mishra 6dedcbddb0 Set correct MIME Type for all types of files served from techdocs-backend 2020-12-15 19:28:50 +01:00
Patrik Oldsberg 1da7d2c657 plugins,packages: replace usages of ConfigReader.from 2020-12-15 19:04:50 +01:00
Himanshu Mishra 320bb9c671 1. Use File.exists and fs.access to check if file exists instead of reading file
2. Clarify name of extension types which need special headers
3. More tests
2020-12-10 23:41:24 +01:00
Himanshu Mishra 9507d72a38 1. Replace klaw with recursive-readdir
2. Break out reading files logic to a helper function (add tests)
3. Use Promise.all to make sure all files are uploaded
2020-12-10 00:15:00 +01:00