Commit Graph

118 Commits

Author SHA1 Message Date
Himanshu Mishra 1ca98d7df8 TechDocs: Use mockFs for tests instead of real filesystem 2021-02-02 23:50:23 +01:00
Himanshu Mishra ae893e99f9 TechDocs: Update tests to reflect requestUrl change 2021-02-02 13:52:47 +01:00
Himanshu Mishra 5d94132bf9 TechDocs: Suppress ts error of config not being used 2021-02-02 13:51:05 +01:00
Himanshu Mishra 3927e79c29 techdocs: local publisher should directly access file system 2021-02-02 13:51:05 +01:00
Himanshu Mishra 38469b66bf techdocs: fix tests by reusing the instantiated discovery 2021-02-02 13:51:05 +01:00
Parth Shandilya e44925723e TechDocs: Make requestUrl and storageUrl optional configs by using discovery APIs
closes #3715

Co-authored-by: Himanshu Mishra <himanshu@orkohunter.net>
2021-02-02 13:51:04 +01:00
Himanshu Mishra 3bdd6eac07 techdocs: Fix broken tests on Windows
https://github.com/backstage/backstage/runs/1805649470\?check_suite_focus\=true
2021-02-01 13:36:46 +01:00
Himanshu Mishra 8eb862c58e Merge pull request #4017 from viavarejo/feature/techdocs-azure-storage 2021-01-31 16:03:24 +01:00
vitorgrenzel a2feeb288d feat(techdocs-common): update tests Azure Blob Storage 2021-01-28 16:17:30 -03:00
vitorgrenzel 9197ac3a0b feat(techdocs-common): update tests Azure Blob Storage 2021-01-28 16:17:29 -03:00
vitorgrenzel 0024d6d8c0 feat(techdocs-common): update Azure Blob Storage 2021-01-28 16:17:28 -03:00
vitorgrenzel 0be9694aa6 feat(techdocs-common): update Azure Blob Storage 2021-01-28 16:17:27 -03:00
vitorgrenzel 2d8d7697d6 feat(techdocs-common): update Azure Blob Storage 2021-01-28 16:17:20 -03:00
vitorgrenzel 59b8d5a0d9 feat(techdocs-common): add Azure Blob Storage 2021-01-28 16:16:22 -03:00
vitorgrenzel c777df180a feat(techdocs-common): add Azure Storage 2021-01-28 16:16:21 -03:00
Himanshu Mishra 03a2d95c71 TechDocs: Update tests with mock url reader 2021-01-28 15:32:09 +01:00
Himanshu Mishra 4bc89c555a TechDocs: Dir preparer should also use URL Reader 2021-01-28 14:54:48 +01:00
github-actions[bot] cda741034d Version Packages 2021-01-28 10:46:08 +00:00
Alan Crosswell 73c299c9ee upgrade git-url-parse to 11.4.4 2021-01-27 11:54:03 -05:00
Matei David 27f529afb6 Rename limitFactory to createLimiter in p-limit import
Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-25 11:43:03 +00:00
Matei David 36bd49fa56 Remove async keyword from anon limiter fn
Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-25 11:40:47 +00:00
Matei David db2328c885 Introduce execution limiter for file uploads
* TechDoc publishers for storage providers (i.e aws, google storage)
 should now rate limit concurrent executions allowing for file uploads to run in batches of 10.
 * Add PATCH changeset to techdocs common
 * Update yarn.lock
2021-01-25 11:40:47 +00:00
github-actions[bot] c5584b62c2 Version Packages 2021-01-21 13:13:05 +00:00
Himanshu Mishra 25f0b76c0b Merge pull request #3968 from mateiidavid/create-techdocs-metadata-type 2021-01-20 21:23:23 +01:00
Himanshu Mishra e8e6bd92b6 Merge pull request #4120 from backstage/orkohunter/url-reader-readTree-with-sha 2021-01-20 14:49:47 +01:00
Himanshu Mishra 060fd0a44a Merge pull request #4109 from backstage/iameap/stream-from-gcs 2021-01-19 12:15:19 +01:00
Himanshu Mishra 8565ad2279 1. Use etag over sha for the identifier
2. Re-combine Resonse types into one
2021-01-18 22:52:45 +01:00
Himanshu Mishra 2305296073 backend-common: Support SHA based caching in URL Reader's readTree
readTree now takes a SHA in options, and throws a NotModifiedError if the SHA matches with the latest commit SHA on the tree. readTree response now contains a new 'sha' parameter.

Also, the default branch will now be used if no branch is provided in the url's target
2021-01-18 21:53:49 +01:00
Eric Peterson da6a29b6d9 Ensure error handling happens the same way. 2021-01-18 18:48:34 +01:00
Matei David a5e27d5c1d Introduce techdocs metadata in techdocs-common
* Add TechdocsMetadata type in backend plugin endpoint
 * Introduce TechDocsMetadata type in frontend
 * Add changeset
 * Remove old thennable metadata resp
 * Address PR feedback
  - Remove explicit type annotation on TechDocsMetadata
  - Reintroduce res.send instead of throwing an error
  - Change logger info to logger error
 * Add TechDocsMetadata type in frontend plugin
 * Commit yarn.lock
 * Introduce JSON5 and remove parsing in local pub
Signed-off-by: Matei David <matei.david.35@gmail.com>
2021-01-16 18:33:53 +00:00
Eric Peterson ce9d5ac220 Remove unnecessary chunk transfer encoding header. 2021-01-15 19:17:01 +01:00
Eric Peterson 8d950d0ae7 Stream files directly from GCS instead of buffering and sending. 2021-01-15 18:02:37 +01:00
Himanshu Mishra 09d211bff2 techdocs-common: release v0.3.4 2021-01-15 12:45:47 +01:00
Himanshu Mishra a594a72576 techdocs-common: Allow techdocs-cli to import in a non-backstage environment
techdocs-common has a dependency on @backstage/plugin-techdocs-backend. This prohibits techdocs-cli to import it and generate/publish docs on a CI/CD environment to an external storage
2021-01-15 09:26:26 +01:00
github-actions[bot] be332e13ea Version Packages 2021-01-14 13:05:31 +00:00
Fredrik Adelöw 371f67ecd0 techdocs-common: fix to-string breakage of binary files 2021-01-13 10:37:32 +01:00
Himanshu Mishra f454b9f865 TechDocs/AWS: Update tutorial with auth best practices 2021-01-12 12:35:13 +01:00
Himanshu Mishra 16b73697a3 TechDocs/AWS: Enable authentication using env variables and ~/.aws/config shared file
If authentication secrets are provided in app-config.yaml, it will be used. If not, environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION will be used. If not present, ~/.aws/config will be used to read the configs.

https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-environment.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html
https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-region.html
2021-01-12 11:41:00 +01:00
Fredrik Adelöw 1c7771871e Merge pull request #3953 from ayshiff/feature/techdocs-aws-sdkv3
TechDocs: AWS S3 SDK version bump
2021-01-12 10:21:51 +01:00
Himanshu Mishra 3790b585c9 Merge pull request #3966 from backstage/orkohunter/techdocs-cli-uses-techdocs-common 2021-01-11 23:23:26 +01:00
Himanshu Mishra 078987e251 techdocs-common: fix tests for the new generator API 2021-01-11 23:08:30 +01:00
Himanshu Mishra 2165901c00 TechDocs: Generator now requires input and output directory and does not create them
The creation of temporary directories and their clean up should be handled on its own. techdocs-cli already has a fixed input output dir requirement. It makes that generator accepts these values as requirements.
2021-01-11 22:33:10 +01:00
Himanshu Mishra 15cc6ff12c Merge pull request #3981 from backstage/orkohunter/improve-gcs-credentials-handling 2021-01-11 21:12:55 +01:00
Himanshu Mishra 97716f2b04 Apply suggestions from code review
Co-authored-by: Fredrik Adelöw <freben@gmail.com>
2021-01-11 20:50:20 +01:00
Remi 5826d0973e feat(techdocs): add changeset + refactor 2021-01-11 18:19:28 +01:00
Remi b18efa0553 feat(techdocs): streamToString try/catch 2021-01-11 17:21:50 +01:00
Remi 60cad4c1ab fix(techdocs): getObject stream 2021-01-11 17:21:50 +01:00
Remi 33b1a43c77 fix(techdocs): broken windows tests 2021-01-11 17:21:50 +01:00
Remi 4bff4d123b feat(techdocs): add aws-sdk types 2021-01-11 17:21:50 +01:00
Remi bb8d83018e feat(techdocs): delete old aws sdk mock 2021-01-11 17:21:50 +01:00