Commit Graph

96 Commits

Author SHA1 Message Date
github-actions[bot] c5584b62c2 Version Packages 2021-01-21 13:13:05 +00:00
Himanshu Mishra 25f0b76c0b Merge pull request #3968 from mateiidavid/create-techdocs-metadata-type 2021-01-20 21:23:23 +01:00
Himanshu Mishra e8e6bd92b6 Merge pull request #4120 from backstage/orkohunter/url-reader-readTree-with-sha 2021-01-20 14:49:47 +01:00
Himanshu Mishra 060fd0a44a Merge pull request #4109 from backstage/iameap/stream-from-gcs 2021-01-19 12:15:19 +01:00
Himanshu Mishra 8565ad2279 1. Use etag over sha for the identifier
2. Re-combine Resonse types into one
2021-01-18 22:52:45 +01:00
Himanshu Mishra 2305296073 backend-common: Support SHA based caching in URL Reader's readTree
readTree now takes a SHA in options, and throws a NotModifiedError if the SHA matches with the latest commit SHA on the tree. readTree response now contains a new 'sha' parameter.

Also, the default branch will now be used if no branch is provided in the url's target
2021-01-18 21:53:49 +01:00
Eric Peterson da6a29b6d9 Ensure error handling happens the same way. 2021-01-18 18:48:34 +01:00
Matei David a5e27d5c1d Introduce techdocs metadata in techdocs-common
* Add TechdocsMetadata type in backend plugin endpoint
 * Introduce TechDocsMetadata type in frontend
 * Add changeset
 * Remove old thennable metadata resp
 * Address PR feedback
  - Remove explicit type annotation on TechDocsMetadata
  - Reintroduce res.send instead of throwing an error
  - Change logger info to logger error
 * Add TechDocsMetadata type in frontend plugin
 * Commit yarn.lock
 * Introduce JSON5 and remove parsing in local pub
Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-16 18:33:53 +00:00
Eric Peterson ce9d5ac220 Remove unnecessary chunk transfer encoding header. 2021-01-15 19:17:01 +01:00
Eric Peterson 8d950d0ae7 Stream files directly from GCS instead of buffering and sending. 2021-01-15 18:02:37 +01:00
Himanshu Mishra 09d211bff2 techdocs-common: release v0.3.4 2021-01-15 12:45:47 +01:00
Himanshu Mishra a594a72576 techdocs-common: Allow techdocs-cli to import in a non-backstage environment
techdocs-common has a dependency on @backstage/plugin-techdocs-backend. This prohibits techdocs-cli to import it and generate/publish docs on a CI/CD environment to an external storage
2021-01-15 09:26:26 +01:00
github-actions[bot] be332e13ea Version Packages 2021-01-14 13:05:31 +00:00
Fredrik Adelöw 371f67ecd0 techdocs-common: fix to-string breakage of binary files 2021-01-13 10:37:32 +01:00
Himanshu Mishra f454b9f865 TechDocs/AWS: Update tutorial with auth best practices 2021-01-12 12:35:13 +01:00
Himanshu Mishra 16b73697a3 TechDocs/AWS: Enable authentication using env variables and ~/.aws/config shared file
If authentication secrets are provided in app-config.yaml, it will be used. If not, environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION will be used. If not present, ~/.aws/config will be used to read the configs.

https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-environment.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html
2021-01-12 11:41:00 +01:00
Fredrik Adelöw 1c7771871e Merge pull request #3953 from ayshiff/feature/techdocs-aws-sdkv3
TechDocs: AWS S3 SDK version bump
2021-01-12 10:21:51 +01:00
Himanshu Mishra 3790b585c9 Merge pull request #3966 from backstage/orkohunter/techdocs-cli-uses-techdocs-common 2021-01-11 23:23:26 +01:00
Himanshu Mishra 078987e251 techdocs-common: fix tests for the new generator API 2021-01-11 23:08:30 +01:00
Himanshu Mishra 2165901c00 TechDocs: Generator now requires input and output directory and does not create them
The creation of temporary directories and their clean up should be handled on its own. techdocs-cli already has a fixed input output dir requirement. It makes that generator accepts these values as requirements.
2021-01-11 22:33:10 +01:00
Himanshu Mishra 15cc6ff12c Merge pull request #3981 from backstage/orkohunter/improve-gcs-credentials-handling 2021-01-11 21:12:55 +01:00
Himanshu Mishra 97716f2b04 Apply suggestions from code review
Co-authored-by: Fredrik Adelöw <freben@gmail.com>
2021-01-11 20:50:20 +01:00
Remi 5826d0973e feat(techdocs): add changeset + refactor 2021-01-11 18:19:28 +01:00
Remi b18efa0553 feat(techdocs): streamToString try/catch 2021-01-11 17:21:50 +01:00
Remi 60cad4c1ab fix(techdocs): getObject stream 2021-01-11 17:21:50 +01:00
Remi 33b1a43c77 fix(techdocs): broken windows tests 2021-01-11 17:21:50 +01:00
Remi 4bff4d123b feat(techdocs): add aws-sdk types 2021-01-11 17:21:50 +01:00
Remi bb8d83018e feat(techdocs): delete old aws sdk mock 2021-01-11 17:21:50 +01:00
Remi 7f34d22bdf feat(techdocs): update aws-sdk to v3 2021-01-11 17:21:50 +01:00
Himanshu Mishra de4c3af867 fix(techdocs): Handle images and other binaries correctly in aws/gcs publishers
Closes https://github.com/backstage/backstage/issues/3990
2021-01-10 10:09:42 +01:00
Himanshu Mishra e7954559ab TechDocs/GCS: Remove projectId since it is not needed
I initially thought the GCS Nodejs client would make use of it, since it asked for it in the contructor. However, we may not need them at all. Buckets are gloablly unique anyway.
2021-01-09 16:29:42 +01:00
Himanshu Mishra 1ef5d0f13c TechDocs/GCS: Enable authentication using environemnt variables
GOOGLE_APPLICATION_CREDENTIALS is the environment variable used by the client. Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions automatically provide a default service account and set this env variable.
https://cloud.google.com/docs/authentication/production
2021-01-09 16:07:39 +01:00
Himanshu Mishra c65dc26c10 TechDocs: Generator can be given a pre-created output directory
TechDocs Generator uses a temporary directory to store the generated files. But with techdocs-cli, the generator needs to output in a fixed directory.

Also, make parsedLocationAnnotation optional since it is not a critical requirement for the generator.
2021-01-08 13:42:55 +01:00
github-actions[bot] 28d7f319f4 Version Packages 2021-01-07 10:31:20 +00:00
Remi a87fc11aca tests(techdocs): refactor 2021-01-05 21:31:18 +01:00
Remi 0f40f1934f fix(techdocs): path delimiters 2021-01-05 18:35:23 +01:00
Himanshu Mishra 0b9646ed6a Merge pull request #3794 from ayshiff/feature/techdocs-aws-s3 2021-01-05 17:48:57 +01:00
Ben Lambert cf2c2df7d3 Merge pull request #3834 from erikxiv/fix/techdocs-ghe-apibaseurl
Fix for integration.github.apiBaseUrl configuration not properly overriding apiBaseUrl used by techdocs
2021-01-04 13:18:22 +01:00
blam 5b38fd70d6 chore: it's safe by default now, so we can remove the nesting 2021-01-04 11:39:22 +01:00
blam 7b3c6d9549 chore: fix yarn lock 2021-01-04 11:26:20 +01:00
blam 9ca93c9f7a chore: udpate usage in techdocs-common too to avoid pulling in two versions of js-yam 2021-01-04 11:22:15 +01:00
Remi 1517009d8a refactor(techdocs-common): add try/catch statements + fs-extra 2021-01-03 03:40:21 +01:00
Remi 3aa98a6753 feat(techdocs): update dependency versions 2020-12-30 10:33:11 +01:00
Himanshu Mishra dc99b07447 fix(techdocs-common) broken test
also copy yarn.lock from master; and yarn again

fix(techdocs-common): broken async test
2020-12-30 10:25:58 +01:00
Remi 1582b03b21 fix(techdocs): fix JSON schema to match AWS S3 region + fix tests 2020-12-30 10:25:57 +01:00
Himanshu Mishra e4dfaba3c1 techdocs: fix metadata API response from S3 storage 2020-12-30 10:25:57 +01:00
Remi 80b60862af feat(techdocs): updates aws variables access 2020-12-30 10:25:57 +01:00
Remi 0661c4f725 fix(techdocs-common): format 2020-12-30 10:25:57 +01:00
Remi b9e6d305e2 fix(techdocs): typos + add tests + requested changes 2020-12-30 10:25:57 +01:00
Remi 134888fb21 fix(techdocs): format + doc quality check 2020-12-30 10:25:57 +01:00