$ curl -X DELETE https://CLUSTER_DOMAIN/v1/admin/scms/SCM_CONFIG_ID \
- -H 'Authorization: _Simple|Bearer_ CLUSTER_ADMIN_AUTH_TOKEN' \
+ -H 'Authorization: Simple|Bearer CLUSTER_ADMIN_AUTH_TOKEN' \
-H 'Content-type: application/json'
@@ -328,7 +328,7 @@ To create an algorithm through the API, begin by adding your algorithm metadata
$ curl -X POST -k https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER \
- -H 'Authorization: _Simple|Bearer_ ORG_ADMIN_AUTH_TOKEN \
+ -H 'Authorization: Simple|Bearer ORG_ADMIN_AUTH_TOKEN \
-H 'Content-type: application/json' \
-d @create_algo_on_bitbucket.json
@@ -426,7 +426,7 @@ Note that in the REST request in **Option A**, below, the algorithm UUID (`ALGO_
$ curl https://api.CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
- -H 'Authorization: _Simple|Bearer_ ORG_ADMIN_AUTH_TOKEN'
+ -H 'Authorization: Simple|Bearer ORG_ADMIN_AUTH_TOKEN'
diff --git a/_pages/algorithm-development/create-scm-backed-algo-api.md b/_pages/algorithm-development/create-scm-backed-algo-api.md
new file mode 100644
index 000000000..b4dc8ba78
--- /dev/null
+++ b/_pages/algorithm-development/create-scm-backed-algo-api.md
@@ -0,0 +1,143 @@
+---
+categories: algorithm-development
+layout: article
+title: Create an Algorithm Backed by an External SCM Provider Through the API
+---
+
+When you create an algorithm through the API, Algorithmia's internal SCM provider is used by default unless an external SCM configuration is specified explicitly. To use an external SCM provider as the source code repository host for your algorithms, a cluster admin must first set up the configuration. At a high level, the workflow looks like this:
+
+On the cluster admin side:
+
+1. [Set up an external SCM configuration](/developers/administration/admin-panel/source-code-management/#creating-scm-configurations) and enable it to back algorithms (this is a one-time setup step).
+2. [List the available SCM configurations](/developers/administration/scm-config/source-code-management-bitbucket-server-api/#listing-scm-configurations) to get the `id` of the configuration of interest.
+
+On the user side:
+
+1. Authorize the SCM configuration through the **Settings** tab on your account profile, under **Source Code Management**.
+2. Create an algorithm, specifying the SCM provider's `id` from above, as well as other required parameters, under the `source` parameter in the request payload.
+
+On this page, we'll walk through the flow on the user's side only. For details on the steps that a cluster admin must take, see the links above.
+
+## 1. Authorize an external SCM configuration
+
+In the UI, go to your account profile and click the **Settings** tab. Scroll to the **Source Code Management** section, where you'll see a list of SCM configurations. Click **Authorize** on the entry that you'd like to authorize; this will enable that SCM configuration to host source code for new algorithms that you create using this account. A popup window will appear, and you'll need to confirm that you authorize the external service.
+
+## 2. Create an algorithm backed by an external SCM
+
+In order to create the algorithm, you'll need the `id` of the SCM configuration, which references in the code below as `SCM_CONFIG_ID`. A cluster admin will need to provide this value, as described above. You'll then follow the standard process for creating an algorithm, but you'll specify some additional information to indicate that you'd like the algorithm to be backed by this specific external SCM, meaning that the source code will be hosted on the external platform and not in Algorithmia's internal SCM server.
+
+### Checking your account's SCM authorization status
+
+Before attempting to create your algorithm, it's a good idea to first ensure that the SCM configuration and authorization process has been completed successfully. If the request below returns `{"authorization_status":"unauthorized"}`, you'll need to re-check in the UI to ensure that you authorized the correct configuration (there may be multiple configurations on your cluster for any given SCM provider).
+
+**REST request**
+
+```bash
+$ curl -X POST https://CLUSTER_DOMAIN/v1/scms/SCM_CONFIG_ID/token/status \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -H 'Content-type: application/json'
+```
+
+**REST response**
+
+```json
+{
+ "scm_username": "SCM_USERNAME",
+ "authorization_status": "authorized",
+ "scm_organizations": [
+ {
+ "scm_org_username": "SCM_ORG_USERNAME",
+ "access_level": "MEMBER"
+ }
+ ]
+}
+```
+
+To make sense of this response, suppose that the `SCM_CONFIG_ID` used in the request above was that of a GitHub configuration. The `scm_username` returned would be the GitHub username authorized through the Algorithmia account, and the `scm_org_username` would be the GitHub account of the organization in which the SCM configuration was created by the Algorithmia cluster admin. Also note that in the response, the string `MEMBER` is not a placeholder, but is actually the string that is returned, indicating that the user is a member of the given organization and has corresponding resource access privileges.
+
+### Creating an algorithm
+
+The algorithm request payload shown below has several `UPPERCASE` placeholders that you'll need to replace in your request. In particular, you'll want to make sure that you:
+
+* Replace the `SCM_CONFIG_ID` value with the one used above, which you've ensured is authorized on the cluster.
+* Replace the `SCM_USERNAME` value (to specify `repository_owner`) with the value returned from the request above to get the SCM authorization status.
+* For `repository_name`, you can choose whatever name makes the most sense to you. Often, it will be clearest to name the repository with the same name as the algorithm (as shown in the code below), but you have the option to name it differently if you choose. Note that you'll need to substitute `ENV_ID` with a valid `id` of an algorithm environment on your cluster. You can find an appropriate `id` value by [listing available languages and environments](/developers/algorithm-development/environments#option-b-using-rest-requests-directly).
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -H 'Content-Type: application/json' \
+ -d '{
+ "name": "ALGO_NAME",
+ "details": {
+ "summary": "Summary of what algorithm does.",
+ "label": "ALGO_NAME",
+ "tagline": "Tag line for algorithm."
+ },
+ "settings": {
+ "source_visibility": "closed",
+ "username": "ALGO_OWNER",
+ "algorithm_environment": "ENV_ID",
+ "license": "apl",
+ "network_access": "isolated",
+ "pipeline_enabled": true
+ },
+ "source": {
+ "repository_name": "ALGO_NAME",
+ "repository_owner": "SCM_USERNAME",
+ "scm": "SCM_CONFIG_ID",
+ "initial_commit_message": "initial commit message"
+ }
+ }'
+```
+
+**REST response**
+
+```json
+{
+ "id": "c6868815-1141-b894-b5e6-ba64f51d723a",
+ "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME"
+ },
+ "settings": {
+ "algorithm_callability": "private",
+ "source_visibility": "closed",
+ "package_set": "python37",
+ "license": "apl",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "insights_enabled": false,
+ "algorithm_environment": "ENV_ID"
+ },
+ "source": {
+ "scm": {
+ "default": false,
+ "enabled": true,
+ "id": "SCM_CONFIG_ID",
+ "name": "https://github.com/",
+ "provider": "github",
+ "scm": {
+ "client_id": "a72cb1fe54e43b70cdb6"
+ },
+ "oauth": {
+ "client_id": "a72cb1fe54e43b70cdb6"
+ },
+ "urls": {
+ "web": "https://github.com/",
+ "api": "https://api.github.com/",
+ "ssh": "ssh://git@github.com"
+ }
+ },
+ "repository_owner": "SCM_USERNAME",
+ "repository_name": "ALGO_NAME",
+ "repository_https_url": "https://github.com/SCM_USERNAME/ALGO_NAME",
+ "repository_ssh_url": "ssh://git@github.com:SCM_USERNAME/ALGO_NAME.git"
+ },
+ "resource_type": "algorithm"
+}
+```
+
+Note that this is an example response for an algorithm created using GitHub as an SCM provider. If you use Bitbucket or GitLab as your provider, provider-specific parameters such as the name and associated URLs will differ.
diff --git a/_pages/algorithm-development/retrieve-build-logs.md b/_pages/algorithm-development/retrieve-build-logs.md
new file mode 100644
index 000000000..41c96c39e
--- /dev/null
+++ b/_pages/algorithm-development/retrieve-build-logs.md
@@ -0,0 +1,152 @@
+---
+categories: algorithm-development
+layout: article
+title: Retrieve Algorithm Build-Time Logs
+---
+
+You can query Algorithmia's API to retrieve build logs for a given algorithm. We offer the following two routes for retrieving build logs; you may decide that one is more convenient based on your workflow or use case. The `/builds` endpoint can be used to retrieve information for any algorithm build, while the `/versions` endpoint will only list algorithm builds that've been published.
+
+* `/v1/algorithms/ALGO_OWNER/ALGO_NAME/builds/BUILD_ID`
+* `/v1/algorithms/ALGO_OWNER/ALGO_NAME/versions/ALGO_VERSION_HASH`
+
+Note that in the code samples below, `AUTH_TOKEN` represents either a standard (non-admin) API key, or a JSON Web Token (JWT), associated with the account that owns the algorithm of interest or that's a member of the organization that owns the algorithm.
+
+Retrieving build logs from the `/builds` endpoint
+-------------------------------------------------
+
+To retrieve the logs for a specific build of an algorithm, you'll first need to know the [algorithm build UUID](/developers/glossary#algorithm-build-uuid). For a given algorithm, you can list all of the algorithm builds with the following request.
+
+**REST request**
+
+```bash
+curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME/builds \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+{
+ "marker": null,
+ "next_link": null,
+ "results": [
+ {
+ "build_id": "d01d403d-a853-4831-9fff-9783a7b7e956",
+ "status": "succeeded",
+ "commit_sha": "8e36fbf53f57fa1f1b681b1f1ab178b6f989976f",
+ "started_at": "2021-07-29T21:12:56.442Z",
+ "finished_at": "2021-07-29T21:13:44.251Z"
+ },
+ {
+ "build_id": "8a366318-3ee9-4a0e-8fc9-7e1af60c67bf",
+ "status": "succeeded",
+ "commit_sha": "0b1e586f4db06756775795bcd4b42ad4352b652e",
+ "started_at": "2021-07-28T18:11:42.749Z",
+ "finished_at": "2021-07-28T18:12:30.946Z",
+ "version_info": {
+ "semantic_version": "0.1.0"
+ }
+ }
+ ]
+}
+```
+
+The `build_id` value in the response is the build UUID. You can use any `build_id` value to replace the value of `BUILD_ID` in the following request to retrieve the logs for a specific algorithm build.
+
+**REST request**
+
+```bash
+curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME/builds/BUILD_ID/logs \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+{
+ "logs": "Sending build context to Docker daemon 1.039MB\n\nStep 1/21 : FROM hub.CLUSTER_DOMAIN/algorithmiahq/langpack-builder:d8f3110a-ad46-4008-a099-a33824522d09 as builder\nd8f3110a-ad46-4008-a099-a33824522d09: Pulling from algorithmiahq/langpack-builder\n83ee3a23efb7: Already exists\n...\nDeleted: sha256:b270ba38027a1071cd8077373357e220d8bf55c2199c1cce574a76a411b11e73\nDeleted: sha256:866ca1d71d949043729ec119130ac3e1785fae8ae3a6c1b3193ddfbc9be7a0dc"
+}
+```
+
+Note that the log output shown above has been shortened for this documentation. The response will contain the entire log output from the algorithm build process, including any errors that may have been encountered.
+
+## Retrieving build logs from the `/versions` endpoint
+
+To retrieve the logs for a specific build of a published algorithm, you'll first need to know its [algorithm version hash](/developers/glossary#algorithm-version-hash). In Algorithmia's browser UI, the [algorithm version hash](/developers/glossary#algorithm-version-hash) information is available under an algorithm profile's **Builds** tab. You can also list all of the algorithm's published versions with the following request.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME/versions \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+**REST response**
+
+```json
+{
+ "marker": null,
+ "next_link": null,
+ "results": [
+ {
+ "id": "4a1b7a2a-4183-4939-a292-26e677aeba73",
+ "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME",
+ "tagline": ""
+ },
+ "settings": {
+ "algorithm_callability": "private",
+ "source_visibility": "closed",
+ "package_set": "python39",
+ "license": "apl",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "insights_enabled": false,
+ "algorithm_environment": "d8f3110a-ad46-4008-a099-a33824522d09"
+ },
+ "version_info": {
+ "git_hash": "8e36fbf53f57fa1f1b681b1f1ab178b6f989976f",
+ "sample_input": "\"\"",
+ "sample_output": "Exception encountered while running sample input",
+ "version_uuid": "ec329139-fca8-4f07-b470-f76521831e27"
+ },
+ ....
+ },
+ ...
+ ]
+}
+```
+
+You can use a version's `git_hash` value from the response to replace `ALGO_VERSION_HASH` in the request below to retrieve the logs. The logs will be returned in the `compilation.output` property of the response body, as shown below.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME/versions/ALGO_VERSION_HASH \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+{
+ "id": "4a1b7a2a-4183-4939-a292-26e677aeba73",
+ "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME",
+ "tagline": ""
+ },
+ ...,
+ "compilation": {
+ "successful": true,
+ "output": "Sending build context to Docker daemon 1.039MB\n\nStep 1/21 : FROM hub.CLUSTER_DOMAIN/algorithmiahq/langpack-builder:d8f3110a-ad46-4008-a099-a33824522d09 as builder\nd8f3110a-ad46-4008-a099-a33824522d09: Pulling from algorithmiahq/langpack-builder\n83ee3a23efb7: Already exists\n...\nDeleted: sha256:b270ba38027a1071cd8077373357e220d8bf55c2199c1cce574a76a411b11e73\nDeleted: sha256:866ca1d71d949043729ec119130ac3e1785fae8ae3a6c1b3193ddfbc9be7a0dc"
+ },
+ "resource_type": "algorithm"
+}
+```
+
+Note that the log output shown above has been shortened for this documentation. The response will contain the entire log output from the algorithm build process, including any errors that may have been encountered.
\ No newline at end of file
diff --git a/_pages/algorithm-development/retrieve-error-logs.md b/_pages/algorithm-development/retrieve-error-logs.md
new file mode 100644
index 000000000..74ce4036b
--- /dev/null
+++ b/_pages/algorithm-development/retrieve-error-logs.md
@@ -0,0 +1,108 @@
+---
+categories: algorithm-development
+layout: article
+title: Retrieve Algorithm Run-Time Error Logs
+---
+
+You can query Algorithmia's API to retrieve error-log stack traces for a given algorithm. The response will contain errors from the past 7 days, constrained to 200 algorithm executions. In order to support a variety of automated workflow configurations, we provide two separate ways to identify an algorithm on your cluster in order to request the logs.
+
+* `/v1/algorithms/ALGO_OWNER/ALGO_NAME/errors`
+* `/v1/algorithms/ALGO_ID/errors`
+
+Note that in the code samples below, `AUTH_TOKEN` represents either:
+
+* A standard (non-admin) API key with [Management Access](/developers/algorithm-development/algorithm-management) turned on, associated with the account that owns the algorithm or that's a member of the organization that owns the algorithm
+* A JSON Web Token (JWT) of a user who's a member of the org that owns the algorithm
+
+## Retrieving error logs using algorithm owner and algorithm name
+
+To retrieve errors using this route, you can use the command below.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME/errors \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+[
+ {
+ "created_at": 1627571664000,
+ "request_id": "req-ce624eee-28e2-4cef-87ee-c22c797079f6",
+ "username": "ALGO_OWNER",
+ "algoname": "ALGO_NAME",
+ "algoversion": "0.1.0",
+ "input": "A payload",
+ "error": "error:\nAll I do is throw an error\nstacktrace:\nSome(Traceback (most recent call last):\n File \"/usr/local/bin/algorithmia-pipe\", line 40, in get_response\n result = call_algorithm(request, algorithm)\n File \"/usr/local/bin/algorithmia-pipe\", line 90, in call_algorithm\n return algorithm.apply(data)\n File \"./src/testError.py\", line 6, in apply\n raise Exception(\"All I do is throw an error\")\nException: All I do is throw an error)",
+ "error_type": "AlgorithmError",
+ "billable_to": "ALGO_OWNER",
+ "worker": "172.41.7.115"
+ },
+ ...
+]
+```
+
+## Retrieving error logs using algorithm UUID
+
+To use this endpoint you must first know the algorithm's UUID, which can be retrieved through a `GET` request to the `algorithms` endpoint, specifying the `ALGO_OWNER` and `ALGO_NAME` as shown in the section above. We provide this route to support workflows in which the values for algorithm owner and algorithm name are unknown and only the algorithm UUID is known. For the purpose of demonstration, however, in this code we're retrieving the UUID using the algorithm name and owner.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+{
+ "id": "4a1b7a2a-4183-4939-a292-26e677aeba73", "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME",
+ "tagline": ""
+ },
+ "settings": {
+ ...
+ },
+ "version_info": {
+ ...
+ },
+ ...
+}
+```
+
+You can use the algorithm's `id` value from the response above to replace `ALGO_ID` in the request below to retrieve the errors.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_ID/errors \
+ -H 'Authorization: Simple AUTH_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**REST response**
+
+```json
+[
+ {
+ "created_at": 1627571664000,
+ "request_id": "req-ce624eee-28e2-4cef-87ee-c22c797079f6",
+ "username": "ALGO_OWNER",
+ "algoname": "ALGO_NAME",
+ "algoversion": "0.1.0",
+ "input": "A payload",
+ "error": "error:\nAll I do is throw an error\nstacktrace:\nSome(Traceback (most recent call last):\n File \"/usr/local/bin/algorithmia-pipe\", line 40, in get_response\n result = call_algorithm(request, algorithm)\n File \"/usr/local/bin/algorithmia-pipe\", line 90, in call_algorithm\n return algorithm.apply(data)\n File \"./src/testError.py\", line 6, in apply\n raise Exception(\"All I do is throw an error\")\nException: All I do is throw an error)",
+ "error_type": "AlgorithmError",
+ "billable_to": "ALGO_OWNER",
+ "worker": "172.41.7.115"
+ },
+ ...
+]
+```
diff --git a/_pages/algorithm-development/your-first-algo.md b/_pages/algorithm-development/your-first-algo.md
index 8c4727418..303999cad 100644
--- a/_pages/algorithm-development/your-first-algo.md
+++ b/_pages/algorithm-development/your-first-algo.md
@@ -31,6 +31,7 @@ Table of contents
- [Developing your algorithm in the Web IDE](#developing-your-algorithm-in-the-web-ide)
- [Publishing your algorithm](#publishing-your-algorithm)
- [Calling your algorithm](#calling-your-algorithm)
+- [Quick Reference: Local Development vs. Web IDE](#quick-reference-local-development-vs-web-ide)
- [Next steps](#next-steps)
- [Language guides](#language-guides)
- [Connecting to data](#connecting-to-data)
@@ -488,6 +489,23 @@ $ch = curl_init();
{% endraw %}
+## Quick Reference: Local Development vs. Web IDE
+
+Every developer has a different workflow, and Algorithmia is built with flexibility in mind. When you're developing on our platform, you may find that you use your local environment for the majority of your work but that sometimes it's convenient to jump into the Web IDE, for example to make minor changes.
+
+Both local and Web IDE development are backed by Git version control, so you can use a normal Git workflow as you would with any other remote repositories. Note that, just like in any other distributed version control scenario, if you use both a local environment and the Web IDE to make changes in the same file(s) at the same time, you may end up with merge conflicts. If you do commit and push code locally via Git while the Web IDE is open to the same repository, simply reload the page to view the latest version.
+
+The following table provides a quick-reference comparison for understanding how algorithm management actions completed using the local workflow map to actions in the browser UI.
+
+|Action|Git and/or Python Client|Browser UI/Web IDE|
+|--- |--- |--- |
+|Create an algorithm|`algo.create(...)`|**Create New** button → **Algorithm** button|
+|Save an algorithm change without a commit|("save/write file" command)|(no equivalent functionality)|
+|Save an algorithm change with a commit|`git commit`|**Save** button|
+|Build an algorithm|`algo.compile()` / `git push`|**Build** button (also creates a commit)|
+|Publish an algorithm|`algo.publish(...)`|**Publish** button|
+|Call an algorithm|`algo.pipe(...)`|(input directly in test console)|
+
## Next steps
Now that you've published your first simple algorithm on the platform, you're empowered to start developing much more complex and useful algorithms. You can write them from scratch, adopt and modify open-source algorithms, and even pipeline multiple algorithms together. Once published, your algorithms will become versioned API endpoints that you can consume from external applications with just a few lines of code!
diff --git a/_pages/clients/cli.md b/_pages/clients/cli.md
index 75a5cbd5d..6b12c4b7f 100644
--- a/_pages/clients/cli.md
+++ b/_pages/clients/cli.md
@@ -111,9 +111,19 @@ You can find more details about optional flags under [API Docs](/developers/api/
### Limits
-By default, one account can make up to {{site.data.stats.platform.max_num_algo_requests}} Algorithmia requests at the same time (this limit can be increased if needed).
+By default, one account can make up to {{site.data.stats.platform.max_num_algo_requests}} concurrent [algorithm execution requests](/developers/glossary#algorithm-execution-request) (this limit can be increased if needed).
-Requests are limited to a payload size of 10 MB for input and 15 MB for output. If you need to work with larger payloads, you can make use of Algorithmia's [data API](/developers/api/#data). See [considerations for transferring large data payloads](https://training.algorithmia.com/using-data-sources/688899#considerations-for-transferring-large-data-payloads) for more details.
+Algorithmia has a 10-MB maximum input data payload limit and a 15-MB maximum output data payload limit. Therefore, to transfer large image files into and out of your algorithm, you may need to upload your data (JSON, JPEG, etc.) to a hosted data collection, passing the data URI to the algorithm as input instead of passing the data itself, and then reading the data from within the algorithm. See Algorithmia's [data API docs](/developers/api/#data) for sample code.
+
+Note that these limits are actually slightly nuanced. When you send an input payload to an algorithm, that data is base64-encoded by the Algorithmia API client. So, for example, a 5-MB sequence of the letter "a" actually ends up as 6 MB of data transferred. You can verify this in a Python shell as shown below.
+
+```python
+>>> import base64
+>>> len(base64.standard_b64encode(b"a"*1024*1024*5))/1024/1024
+6.666667938232422
+```
+
+If your payload has unicode characters, they occupy several bytes each. When these characters are encoded, the discrepancy in payload size will be even larger than that shown above. Therefore, if sending and receiving large payloads, you might run up against these limits, even if the file size itself is not greater than the exact input or output limits.
## Working with Algorithmia data sources
@@ -210,9 +220,9 @@ $ algo run ALGO_OWNER/ALGO_NAME -d "HAL 9000" --profile second_profile
Hello HAL 9000
```
-## List algorithm languages and environments and download algorithm template files
+## Listing algorithm languages and environments and downloading algorithm template files
-When you create an algorithm, you can choose to use a predefined [algorithm environment](/developers/algorithm-development/environments) that's been optimized for the Algorithmia platform with specific ML library dependencies baked in. You can use the algo CLI to list the algorithm languages on a specific cluster, select a language and list its corresponding algorithm environments, and then download algorithm template files if desired. The algo CLI makes this multi-step process straightforward, and the steps are documented in our [Training Center](https://training.algorithmia.com/developing-python-algorithms-in-a-local-ide/855746).
+When you create an algorithm, you can choose to use a predefined [algorithm environment](/developers/algorithm-development/environments) that's been optimized for the Algorithmia platform with specific ML library dependencies baked in. You can [use the algo CLI to list the algorithm languages](/developers/algorithm-development/environments/#list-languages-and-environments-and-download-algorithm-template-files) on a specific cluster, select a language and list its corresponding algorithm environments, and then download algorithm template files if desired. The algo CLI makes this multi-step process straightforward.
## Additional functionality
diff --git a/_pages/clients/curl.md b/_pages/clients/curl.md
index f04d24483..64268cdf3 100644
--- a/_pages/clients/curl.md
+++ b/_pages/clients/curl.md
@@ -96,7 +96,17 @@ To learn how to handle errors elagantly and expressively in your own algorithms,
By default, one account can make up to {{site.data.stats.platform.max_num_algo_requests}} concurrent [algorithm execution requests](/developers/glossary#algorithm-execution-request) (this limit can be increased if needed).
-Requests are limited to a payload size of 10 MB for input and 15 MB for output. If you need to work with larger payloads, you can make use of Algorithmia's [data API](/developers/api/#data). See [considerations for transferring large data payloads](https://training.algorithmia.com/using-data-sources/688899#considerations-for-transferring-large-data-payloads) for more details.
+Algorithmia has a 10-MB maximum input data payload limit and a 15-MB maximum output data payload limit. Therefore, to transfer large image files into and out of your algorithm, you may need to upload your data (JSON, JPEG, etc.) to a hosted data collection, passing the data URI to the algorithm as input instead of passing the data itself, and then reading the data from within the algorithm. See Algorithmia's [data API docs](/developers/api/#data) for sample code.
+
+Note that these limits are actually slightly nuanced. When you send an input payload to an algorithm, that data is base64-encoded by the Algorithmia API client. So, for example, a 5-MB sequence of the letter "a" actually ends up as 6 MB of data transferred. You can verify this in a Python shell as shown below.
+
+```python
+>>> import base64
+>>> len(base64.standard_b64encode(b"a"*1024*1024*5))/1024/1024
+6.666667938232422
+```
+
+If your payload has unicode characters, they occupy several bytes each. When these characters are encoded, the discrepancy in payload size will be even larger than that shown above. Therefore, if sending and receiving large payloads, you might run up against these limits, even if the file size itself is not greater than the exact input or output limits.
## Working with Algorithmia data sources
diff --git a/_pages/datarobot.md b/_pages/datarobot.md
new file mode 100644
index 000000000..32df79f97
--- /dev/null
+++ b/_pages/datarobot.md
@@ -0,0 +1,6 @@
+---
+layout: article
+title: DataRobot MLOps
+---
+
+Please see DataRobot's [MLOPs platform docs](https://docs.datarobot.com/en/docs/mlops/index.html) for documentation on DataRobot's MLOps product.
diff --git a/_pages/event-flows/amazon-sqs.md b/_pages/event-flows/amazon-sqs.md
index 2c07531e5..efeaf81ba 100644
--- a/_pages/event-flows/amazon-sqs.md
+++ b/_pages/event-flows/amazon-sqs.md
@@ -306,12 +306,13 @@ In the body of your algorithm, paste the code below, replacing `COLLECTION_OWNER
```python
import Algorithmia
-
-client = Algorithmia.client()
+from Algorithmia import ADK
def apply(input):
- data_uri = "data://COLLECTION_OWNER/COLLECTION_NAME/" + input.get("filename")
- client.file(data_uri).put(input.get("data"))
+ Algorithmia.client().file("data://COLLECTION_OWNER/COLLECTION_NAME/" + input.get("filename")).put(input.get("data"))
+
+algorithm = ADK(apply)
+algorithm.init("Algorithmia")
```
diff --git a/_pages/event-flows/azure-sb.md b/_pages/event-flows/azure-sb.md
index 81b7b93a1..9f4b311c6 100644
--- a/_pages/event-flows/azure-sb.md
+++ b/_pages/event-flows/azure-sb.md
@@ -233,12 +233,13 @@ In the body of your algorithm, paste the source code below, replacing the `COLLE
```python
import Algorithmia
-
-client = Algorithmia.client()
+from Algorithmia import ADK
def apply(input):
- data_uri = "data://COLLECTION_OWNER/COLLECTION_NAME/" + input.get("filename")
- client.file(data_uri).put(input.get("data"))
+ Algorithmia.client().file("data://COLLECTION_OWNER/COLLECTION_NAME/" + input.get("filename")).put(input.get("data"))
+
+algorithm = ADK(apply)
+algorithm.init("Algorithmia")
```
@@ -326,20 +327,20 @@ In the body of your algorithm, paste the source code below, replacing the `QUEUE
```python
import os
+import json
import Algorithmia
+from Algorithmia import ADK
from azure.servicebus import ServiceBusClient, ServiceBusMessage
-CONNECTION_STR = os.getenv("CONNECTION_STR")
+CONNECTION_STR = "CONNECTION_STR"
QUEUE_NAME = "QUEUE_NAME"
-
def send_single_message(sender, message):
# Send the message to the queue
sender.send_messages(message)
print("Sent a single message.")
-
def apply(input):
"""Send a message to the queue.
@@ -348,7 +349,7 @@ def apply(input):
"""
# Create a Service Bus message
- message = ServiceBusMessage(input.get("data"))
+ message = ServiceBusMessage(json.dumps(input))
# Create a Service Bus client using the connection string
servicebus_client = ServiceBusClient.from_connection_string(
conn_str=CONNECTION_STR,
@@ -360,6 +361,9 @@ def apply(input):
# Send one message
send_single_message(sender, message)
return "Sent a message to the queue."
+
+algorithm = ADK(apply)
+algorithm.init("Algorithmia")
```
Click the **Save** and **Build** buttons, and then **Publish** the algorithm.
diff --git a/_pages/glossary.html b/_pages/glossary.html
index e1026836e..48863f87a 100644
--- a/_pages/glossary.html
+++ b/_pages/glossary.html
@@ -228,7 +228,6 @@
top-level algorithm
topic
trained model
-
Training Center
user
user account
UNILOG
@@ -1276,12 +1275,6 @@
href="#serialized-model" title="" parentId="dd-serialized-model">serialized model.
-
Training Center [deprecated]
-
- Tutorial-based training content has been moved to DataRobot University.
-
-
user
Human interacting with the Algorithmia platform. See identity.
diff --git a/_pages/integrations/datarobot.md b/_pages/integrations/datarobot.md
index a2f806a12..c472c74de 100644
--- a/_pages/integrations/datarobot.md
+++ b/_pages/integrations/datarobot.md
@@ -11,6 +11,9 @@ tags: [integrations, model-training]
title: "DataRobot"
---
+COMING SOON: Improved Java scoring code support and deeper integrations with DataRobot to support seamless deployment of DataRobot models on the Algorithmia platform.
+{: .notice-info}
+
DataRobot Prime enables you to export a [DataRobot](https://www.datarobot.com/)-trained ML model for deployment to production on an external platform such as Algorithmia. To do this, DataRobot Prime creates a series of rules that approximate the original model and then encapsulates these rules into a Python module or Java class that can be exported for use externally.
For details on how to export a model using DataRobot Prime, see this [DataRobot blog post](https://community.datarobot.com/t5/resources/exporting-models-with-datarobot-prime/ta-p/4629).
@@ -22,7 +25,7 @@ algorithmia>=1.0.0,<2.0
six
```
-In this example, the exported DataRobot Prime model is in `MODEL_FILE.jar`, which has been uploaded to the hosted data collection `COLLECTION_NAME`, owned by the Algorithmia account `COLLECTION_OWNER`.
+In this example, the exported DataRobot Prime model is in `MODEL_FILE.jar`, which has been uploaded to the hosted data collection `COLLECTION_NAME`, owned by the Algorithmia account `COLLECTION_OWNER`.
The workflow used in this code is standard for a Python algorithm, with the main exception being that we call the actual model using Java. Specifically, in order to run the JAR file, we shell out to the Java interpreter using the Python standard libary's `subprocess.Popen()` class. Essentially, this provides a Python wrapper for the Java model, enabling data scientists to work in Python but to use the JAR file from DataRobot.
diff --git a/_pages/model-deployment/environment-migration.md b/_pages/model-deployment/environment-migration.md
new file mode 100644
index 000000000..894358980
--- /dev/null
+++ b/_pages/model-deployment/environment-migration.md
@@ -0,0 +1,404 @@
+---
+categories: algorithm-development
+layout: article
+title: Migrate a Legacy-Language Algorithm to an Environment-Based Algorithm
+---
+
+## Table of Contents
+
+- [Table of Contents](#table-of-contents)
+- [Overview](#overview)
+- [Getting an algorithm's details](#getting-an-algorithms-details)
+- [Migrating an algorithm](#migrating-an-algorithm)
+ - [Python](#python)
+ - [Java](#java)
+ - [Scala](#scala)
+ - [R](#r)
+ - [JavaScript, Ruby, and Rust](#javascript-ruby-and-rust)
+
+## Overview
+
+Beginning in Algorithmia version 21.2, support for legacy language-based algorithms is deprecated. This page contains information for migrating a legacy-language based algorithm to the corresponding environment-based language. To learn more about algorithm environments, and to view the environments that we currently have available, see the [Environments](/developers/model-deployment/environments) page. To get a list of the available languages and their associated algorithm environments currently installed on your cluster, see [Listing languages and environments on your cluster](/developers/model-deployment/environments#list-languages-and-environments-and-download-algorithm-template-files).
+
+## Getting an algorithm's details
+
+To begin, retrieve your algorithm's details to determine whether it's based on a legacy language and therefore needs to be migrated.
+
+When you get the algorithm details (see code [below](#rest-request-algorithm-details)) for a legacy-language based algorithm, the REST response will contain a `language` property listing the name of the language. You can also find the language through the browser UI under the algorithm profile's **Settings** tab, in the **Language** field. The following table lists the strings that you'll see for legacy algorithms. (Note that we no longer support [Javascript, Ruby, or Rust](#javascript-ruby-and-rust) algorithms.)
+
+|Language|`language` field in REST response|Language field in browser UI|
+|--- |--- |--- |
+|Python 2|`python2-langpack`|Python 2.x (Legacy)|
+|Python 3|`python3-1`|Python 3.x (Legacy)|
+|Java|`java`|Java (Legacy)|
+|Scala|`scala`|Scala (Legacy)|
+|R|`r`|R (Legacy)|
+
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -H 'Authorization: Simple STD_API_KEY'
+```
+
+As noted above, if the algorithm is **legacy-language based**, the response will contain a `language` property listing the name of the language. It'll also contain an `environment` property listing the type of hardware (CPU or GPU) that the algorithm uses.
+
+**REST response**
+
+```json
+{
+ "id": "3cfdb152-d1f2-4b08-82b4-681c0b055a4f",
+ "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME",
+ "tagline": ""
+ },
+ "settings": {
+ "algorithm_callability": "private",
+ "source_visibility": "closed",
+ "language": "python2-langpack",
+ "environment": "cpu",
+ "license": "apl",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "insights_enabled": false
+ },
+ "source": {
+ "scm": {
+ "id": "internal",
+ "provider": "internal",
+ "default": true,
+ "enabled": true
+ }
+ },
+ "resource_type": "algorithm"
+}
+```
+
+If the algorithm is **environment based**, the response will contain an `algorithm_environment` property, which is a UUID, as well as a `package_set` property, which specifies the bundle of packages included in the environment.
+
+**REST response**
+
+```json
+{
+ "id": "9aa57ed8-c8db-4bce-9cca-0242af442dfe",
+ "name": "ALGO_NAME",
+ "details": {
+ "label": "ALGO_NAME",
+ "tagline": ""
+ },
+ "settings": {
+ "algorithm_callability": "private",
+ "source_visibility": "closed",
+ "package_set": "python27",
+ "algorithm_environment": "717d36e0-222c-44a0-9aa8-06f4ebc1b82a",
+ "license": "apl",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "insights_enabled": false
+ },
+ "source": {
+ "scm": {
+ "id": "internal",
+ "provider": "internal",
+ "default": true,
+ "enabled": true
+ }
+ },
+ "resource_type": "algorithm"
+}
+```
+
+Migrating an algorithm
+----------------------
+
+To begin, verify that you have a legacy-language based algorithm that needs to be migrated in the first place. To do this, [get the algorithm's details](#getting-an-algorithms-details). If you determine that your algorithm is environment based, no migration is necessary. Otherwise, proceed with the instructions below for the language in question.
+
+ - [Python](#python)
+ - [Java](#java)
+ - [Scala](#scala)
+ - [R](#r)
+ - [JavaScript, Ruby, and Rust](#javascript-ruby-and-rust)
+
+### Python
+
+To migrate a legacy Python algorithm, [get the environment ID value](/developers/model-deployment/environments#listing-algorithm-environments-rest) for a valid Python 2 or Python 3 algorithm environment. Note that in the sample code at the link provided, you'll use the string `python2` or `python3`, as appropriate, to replace `ALGO_LANG` to list the available environments. The environment ID will be a UUID like `2be2e978-9deb-494d-a2d1-e07e121d5ba0`. Execute the following `PUT` request to update the algorithm's settings, using this UUID to replace `ENV_ID`.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -X PUT
+ -H 'Content-Type: application/json' \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -d '{
+ "settings":
+ {
+ "source_visibility": "closed",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "algorithm_environment": "ENV_ID",
+ "license":"apl"
+ }
+ }'
+```
+
+### Java
+
+To migrate a legacy Java algorithm, [get the environment ID value](/developers/model-deployment/environments#listing-algorithm-environments-rest) for the Java 11 algorithm environment (there's currently only one). Note that in the sample code at the link provided, you'll use the string `java11` to replace `ALGO_LANG`. The environment ID will be a UUID like `2be2e978-9deb-494d-a2d1-e07e121d5ba0`. Execute the following `PUT` request to update the algorithm's settings, using this UUID to replace `ENV_ID`.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -X PUT
+ -H 'Content-Type: application/json' \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -d '{
+ "settings":
+ {
+ "source_visibility": "closed",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "algorithm_environment": "ENV_ID",
+ "license":"apl"
+ }
+ }'
+```
+
+Now clone your algorithm locally and move your algorithm's main file `ALGONAME.java` to a new package and rename it to `Algorithm.java`, as demonstrated in the following code.
+
+```bash
+$ git clone https://git.CLUSTER_DOMAIN/git/ALGO_OWNER/ALGO_NAME.git
+$ cd ALGO_NAME
+$ mkdir -p src/main/java/com/algorithmia/ALGO_NAME
+$ mv src/main/java/algorithmia/ALGO_NAME/ALGO_NAME.java \
+ src/main/java/com/algorithmia/ALGO_NAME/Algorithm.java
+```
+
+Similarly, if you have test files, move them to a new package as well.
+
+```bash
+$ mkdir -p src/test/java/com/algorithmia/ALGO_NAME
+$ mv src/test/java/algorithmia/ALGO_NAME/ALGO_NAME_test.java \
+ src/test/java/com/algorithmia/ALGO_NAME/
+```
+
+Now make the following changes in your algorithm's [Project Object Model (POM)](https://github.com/algorithmiaio/langpacks/blob/develop/languages/java11/template/pom.xml) file.
+
+* Add the following **[``](https://github.com/algorithmiaio/langpacks/blob/e91077e810da381128ba642ba5240c30c99b1f37/languages/java11/template/pom.xml#L5)**.
+
+```java
+
+ 8
+ 8
+
+```
+
+* Add the following **[``](https://github.com/algorithmiaio/langpacks/blob/e91077e810da381128ba642ba5240c30c99b1f37/languages/java11/template/pom.xml#L45)** on the `Algorithm Handler`.
+
+```java
+
+ com.algorithmia
+ algorithm-handler
+ \[1.2,)
+
+```
+
+* Replace the contents of the **[``](https://github.com/algorithmiaio/langpacks/blob/e91077e810da381128ba642ba5240c30c99b1f37/languages/java11/template/pom.xml#L66)** tag with the following. Note that you'll need to replace `ALGO_NAME` with the name of your algorithm.
+
+```java
+
+
+
+
+ org.apache.maven.plugins
+ maven-dependency-plugin
+
+
+ copy-dependencies
+ prepare-package
+
+ copy-dependencies
+
+
+ ${project.build.directory}/lib
+ false
+ false
+ true
+
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-jar-plugin
+ 3.0.2
+
+
+
+ com.algorithmia.ALGO_NAME.Algorithm
+ true
+ lib/
+
+
+
+
+
+
+```
+
+Once you've added the snippets above to your dependencies file, make the following changes to your source code in [`Algorithm.java`](https://github.com/algorithmiaio/langpacks/blob/e91077e810da381128ba642ba5240c30c99b1f37/languages/java11/template/src/main/java/com/algorithmia/algorithm/Algorithm.java):
+
+* Update the package declaration to `package com.algorithmia.algorithm`.
+* Change the class to extend `AbstractAlgorithm` with appropriate input and output types.
+
+```
+class Algorithm extends AbstractAlgorithm{
+ ...
+}
+```
+
+* Remove `throws Exception` from the `apply` method.
+* Add the following method, renaming the class to simply `Algorithm`.
+
+```
+ public static void main(String[] args) {
+ Algorithm algorithm = new Algorithm();
+ Handler algo = new Handler<>(algorithm);
+ algo.serve();
+ }
+```
+
+You can iterate locally by running `mvn package` while updating the dependency file to ensure that compilation succeeds, and then push the code back up to your Algorithmia cluster.
+
+### Scala
+
+To migrate a legacy Scala algorithm, [get the environment ID value](/developers/model-deployment/environments#listing-algorithm-environments-rest) for the Scala 2 algorithm environment (there's currently only one). Note that in the sample code at the link provided, you'll use the string `scala-2` to replace `ALGO_LANG`. The environment ID will be a UUID like `2be2e978-9deb-494d-a2d1-e07e121d5ba0`. Execute the following `PUT` request to update the algorithm's settings, using this UUID to replace `ENV_ID`.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -X PUT
+ -H 'Content-Type: application/json' \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -d '{
+ "settings":
+ {
+ "source_visibility": "closed",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "algorithm_environment": "ENV_ID",
+ "license":"apl"
+ }
+ }'
+```
+
+Now clone your algorithm locally and move your algorithm's main file `ALGONAME.scala` to a new package and rename it to `Algorithm.scala`, as demonstrated in the following code.
+
+```bash
+$ git clone https://git.CLUSTER_DOMAIN/git/ALGO_OWNER/ALGO_NAME.git
+$ cd ALGO_NAME
+$ mkdir -p src/main/scala/com/algorithmia
+$ mv src/main/scala/algorithmia/ALGO_NAME/ALGO_NAME.scala \
+ src/main/scala/com/algorithmia/Algorithm.scala
+```
+
+Similarly, if you have test files, move them to a new package as well.
+
+```bash
+$ mkdir -p src/test/scala/com/algorithmia
+$ mv src/test/scala/algorithmia/ALGO_NAME/ALGO_NAME_test.scala \
+ src/test/scala/com/algorithmia/
+```
+
+Once you've moved the source files, update **[`build.sbt`](https://github.com/algorithmiaio/langpacks/blob/develop/languages/scala-2/template/build.sbt)** with the following:
+
+* Set `name := "algorithm"`.
+* Set `scalaVersion := "2.13.1"`.
+* Add `enablePlugins(JavaAppPackaging)`.
+* Add `"com.algorithmia" %% "algorithmia-scala" % "1.0.+",` to your `libraryDependencies`.
+* Remove the line containing `mainClass`.
+
+Now add a file **[`project/plugins.sbt`](https://github.com/algorithmiaio/langpacks/blob/develop/languages/scala-2/template/project/plugins.sbt)** with the following contents.
+
+```scala
+addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.4.1")
+```
+
+Once you've added the snippets above to your build and plugins files, make the following changes to your source code in **[`Algorithm.scala`](https://github.com/algorithmiaio/langpacks/blob/develop/languages/scala-2/template/src/main/scala/com/algorithmia/Algorithm.scala)**:
+
+* Update the package declaration to `package com.algorithmia`.
+* Change the class to extend `AbstractAlgorithm` with appropriate input and output types.
+
+```scala
+class Algorithm extends AbstractAlgorithm\[String, String\] {
+ ...
+}
+```
+
+* Update the `apply` method to return `Try[Output type]` instead of just the output type directly.
+* Add the following companion object.
+
+```scala
+object Algorithm {
+ val handler = Algorithmia.handler(new Algorithm)
+
+ def main(args: Array\[String\]): Unit = {
+ handler.serve()
+ }
+}
+```
+
+You can iterate locally by running `sbt stage` to ensure that compilation succeeds, and then push the code back up to your Algorithmia cluster.
+
+### R
+
+To migrate a legacy R algorithm, [get the environment ID value](/developers/model-deployment/environments#listing-algorithm-environments-rest) for a valid R 3.6 algorithm environment. Note that in the sample code at the link provided, you'll use the string `r36`, as appropriate, to replace `ALGO_LANG` to list the available environments. The environment ID will be a UUID like `2be2e978-9deb-494d-a2d1-e07e121d5ba0`. Execute the following `PUT` request to update the algorithm's settings, using this UUID to replace `ENV_ID`.
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithms/ALGO_OWNER/ALGO_NAME \
+ -X PUT
+ -H 'Content-Type: application/json' \
+ -H 'Authorization: Simple STD_API_KEY' \
+ -d '{
+ "settings":
+ {
+ "source_visibility": "closed",
+ "network_access": "full",
+ "pipeline_enabled": true,
+ "algorithm_environment": "ENV_ID",
+ "license":"apl"
+ }
+ }'
+```
+
+To migrate a legacy R algorithm, clone your algorithm locally and rename the algorithm's main file to `Algorithm.r` as demonstrated in the following code.
+
+```bash
+$ git clone https://git.CLUSTER_DOMAIN/git/ALGO_OWNER/ALGO_NAME.git
+$ cd ALGO_NAME
+$ mv src/ALGO_NAME.r src/Algorithm.r
+```
+
+Now add a `load` function to the source code in **[`Algorithm.r`](https://github.com/algorithmiaio/langpacks/blob/develop/languages/r36/template/src/Algorithm.r)** and a call to create the `AlgorithmHandler`.
+
+```r
+load <- function() {
+ NULL
+}
+
+algo <- getAlgorithmHandler(algorithm, load)
+algo$serve()
+```
+
+That's it; simply push your code back up to your Algorithmia cluster and the algorithm will continue to execute without issues into the future.
+
+### JavaScript, Ruby, and Rust
+
+These languages are no longer supported. Please contact your customer success manager if you need support for one of these languages.
\ No newline at end of file
diff --git a/_pages/model-deployment/environments.md b/_pages/model-deployment/environments.md
index be3d6faff..ebb3afc18 100644
--- a/_pages/model-deployment/environments.md
+++ b/_pages/model-deployment/environments.md
@@ -12,7 +12,25 @@ tags: [algo-model-guide]
title: "Algorithm Environments"
---
+- [Overview](#overview)
+- [Why use a predefined environment?](#why-use-a-predefined-environment)
+- [View currently available environments](#view-currently-available-environments)
+ - [Environments available on all clusters](#environments-available-on-all-clusters)
+ - [Environments available by request](#environments-available-by-request)
+- [List languages and environments and download algorithm template files](#list-languages-and-environments-and-download-algorithm-template-files)
+ - [Overview of steps](#overview-of-steps)
+ - [Option A: using the algo CLI](#option-a-using-the-algo-cli)
+ - [Upgrading the algo CLI](#upgrading-the-algo-cli)
+ - [Listing algorithm languages (CLI)](#listing-algorithm-languages-cli)
+ - [Listing algorithm environments (CLI)](#listing-algorithm-environments-cli)
+ - [Downloading template files (CLI)](#downloading-template-files-cli)
+ - [Option B: Using REST requests directly](#option-b-using-rest-requests-directly)
+ - [Listing algorithm languages (REST)](#listing-algorithm-languages-rest)
+ - [Listing algorithm environments (REST)](#listing-algorithm-environments-rest)
+ - [Downloading template files (REST)](#downloading-template-files-rest)
+
## Overview
+
Algorithmia supports seven languages for writing algorithms: Python, R, Java, Scala, Ruby, Rust, and JavaScript. When you create an algorithm, you must select one of these languages.
For several of these languages, we've also created predefined, customized algorithm runtime environments to support various ML frameworks and hardware requirements. These environments are available to both our Enterprise and Teams customers.
@@ -33,11 +51,9 @@ However, we recommend not doing this, and instead choosing the appropriate prede
The algorithm runtime environment you select should match up as closely as possible with your code dependencies, in order to provide the most streamlined development experience and the best algorithm performance. With the base environment selected, you can then add any other required dependencies in whichever way is standard for your language.
-To create an algorithm with a predefined environment through the API, see [Using a predefined language environment](https://training.algorithmia.com/developing-python-algorithms-in-a-local-ide/698050#predefined-environment). The algorithm environments that are currently available are listed below.
-
## View currently available environments
-In the previous lesson, we guided you through how to list the environments available on your cluster for each particular language. On this page, we provide a list of available environments that we've created and that can be made available to you to upon request. Once we provide you with the requested environment, you'll be able to [install it on your cluster](/developers/administration/admin-panel/algorithm-environments/#installing-environments) so that your users can select it when they create new algorithms.
+Below is a list of available environments that we've created and that can be made available to you to upon request. Once we provide you with the requested environment, you'll be able to [install it on your cluster](/developers/administration/admin-panel/algorithm-environments/#installing-environments) so that your users can select it when they create new algorithms. For steps on how to list the environments available on your cluster for each particular language, see [List languages and environments and download algorithm template files](#list-languages-and-environments-and-download-algorithm-template-files).
### Environments available on all clusters
@@ -75,3 +91,221 @@ We're always adding new optimized runtime environments to support our customers
|r36|R 3.6|--|--|--|algorithmiahq/ubuntu:16.04|
|scala-2-sbt-1.3.3|Scala 2.x|--|--|--|algorithmiahq/ubuntu:16.04|
|java11|Java 11.0|--|--|--|algorithmiahq/ubuntu:20.04|
+
+
+## List languages and environments and download algorithm template files
+
+Algorithm template files provide a unified starting point for developing algorithms using a specific algorithm language and environment. This page provides instructions for how to download template files for a specific language and environment using both the algo CLI and REST requests directly. The algo CLI is recommended if you're working at the command line, while we recommend using REST requests directly for automating your workflow.
+
+### Overview of steps
+
+Template files are specific to each algorithm environment, and algorithm environments are specific to each algorithm language. Therefore, to download template files, you'll first need to know what languages and environments are available on your cluster. The general steps are:
+
+1. List the available algorithm languages on your cluster.
+2. List the available environments on your cluster for a language chosen from (1).
+3. Download the template files for an environment chosen from (2).
+
+### Option A: using the algo CLI
+
+#### Upgrading the algo CLI
+
+To use the CLI functionality below, you'll need algo CLI version 1.9.0 or greater. To see what version you currently have installed, you can run `pip list | grep algorithmia`. If you have `algorithmia<1.9.0`, update by running `pip install algorithmia --upgrade`.
+
+#### Listing algorithm languages (CLI)
+
+To list the languages available on your cluster, run the command below, which will list the details for each language.
+
+```bash
+$ algo languages
+Name Display Name
+anaconda3 Conda (Environments) - beta
+csharp-dotnet-core2 C# .NET Core 2.x+ (Environments)
+java11 Java OpenJDK 11.0 (Environments)
+python2 Python 2.x (Environments)
+python3 Python 3.x (Environments)
+r36 R 3.6.x (Environments)
+scala-2 Scala 2.x & sbt 1.3.x (Environments)
+```
+
+#### Listing algorithm environments (CLI)
+
+Environment IDs are unique on every cluster, so you'll need to request the list of environments for your language of choice in order to get the ID value associated with a specific environment. To list environments, run `algo environment ALGO_LANG`, where `ALGO_LANG` is the name of your language of interest from the list [above](#listing-algorithm-languages). This will print out metadata for the available environments. For example, to list Python environments, you can use:
+
+```bash
+$ algo environment python3
+[
+ ...,
+ {
+ "id": "fd980f4f-1f1c-4b2f-a128-d60b40c6567a",
+ "environment_specification_id": "36fd467e-fbfe-4ea6-aa66-df3f403b7132",
+ "display_name": "Python 3.8 + TensorFlow GPU 2.3",
+ "description": "Python 3.8 installation with CUDA 10.1 and TensorFlow 2.3 installed",
+ "created_at": "2021-06-03T19:46:29.111",
+ "language": {
+ "name": "python3",
+ "display_name": "Python 3.x (Environments)",
+ "configuration": "{\n \"display_name\": \"Python 3.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ "machine_type": "GPU"
+ },
+ {
+ "id": "ea8f42fe-ea59-4eab-920a-09176729b4fa",
+ "environment_specification_id": "c6d34b46-0851-455d-97f5-ea8fd8b70552",
+ "display_name": "Python 3.7 + MXNet 1.3.1",
+ "description": "Python 3.7 installation with CUDA 9.0 and MXNet 1.3.1 installed",
+ "created_at": "2021-06-03T18:03:14.704",
+ "language": {
+ "name": "python3",
+ "display_name": "Python 3.x (Environments)",
+ "configuration": "{\n \"display_name\": \"Python 3.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ "machine_type": "GPU"
+ },
+ ...
+]
+```
+
+Depending on your cluster, the above command might return a very long list of environments, many of which won't be useful to you. We recommend filtering the list returned; for example, to only display environments with TensorFlow 2.x preinstalled, you can run:
+
+```bash
+$ algo environment python3 | grep -iC4 -e "tensorflow 2"
+```
+
+#### Downloading template files (CLI)
+
+To download template files for an environment from above, use the following command, replacing `ENV_SPEC_ID` with an environment's `environment_specification_id` value from [above](#listing-algorithm-environments), and `DEST` with the local destination (directory path) to which to save the files (specify a dot "`.`" to save the template files to the current directory). The template files will be saved to the local directory specified and can now be checked in to source control for algorithm development.
+
+```bash
+$ algo template ENV_SPEC_ID DEST
+```
+
+### Option B: Using REST requests directly
+
+To list languages and environments programmatically, you can use REST requests and then parse the JSON directly.
+
+#### Listing algorithm languages (REST)
+
+To list the languages available on your cluster, run the command below, substituting `AUTH_TOKEN` with an authentication token for any account on the cluster. Either specify `Simple` and provide an [API key](https://algorithmia.com/developers/platform/customizing-api-keys) or specify `Bearer` and provide a [JSON Web Token (JWT)](https://algorithmia.com/developers/platform/jwt-authentication). For example, in the code below, for `-H 'Authorization: Simple|Bearer AUTH_TOKEN'` you'd use one or the other of the following commands, replacing the placeholders as appropriate:
+
+* * `'-H Authorization: Simple ADMIN_API_KEY'`
+ * `'-H Authorization: Bearer JSON_WEB_TOKEN'`
+
+**REST request**
+
+```bash
+$ curl https://CLUSTER_DOMAIN/v1/algorithm-environments/edge/languages \
+ -H 'Authorization: Simple|Bearer AUTH_TOKEN' \
+ -H 'Content-type: application/json'
+```
+
+**REST response**
+
+```json
+[
+ {
+ "name":"anaconda3",
+ "display_name":"Conda (Environments) - beta",
+ "configuration":"{\n \"display_name\": \"Conda (Environments) - beta\",\n \"req_files\": \[\n \"environment.yml\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.cache\", \"destination\":\"/home/algo/.cache/\"},\n {\"source\":\"/home/algo/anaconda_environment\", \"destination\": \"/home/algo/anaconda_environment/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ {
+ "name":"csharp-dotnet-core2",
+ "display_name":"C# .NET Core 2.x+ (Environments)",
+ "configuration":"{\n \"display_name\": \"C# .NET Core 2.x+ (Environments)\",\n \"artifacts\": \[\n {\"source\":\"/opt/algorithm/bin/Release/\*/\*\", \"destination\":\"/opt/algorithm/\"},\n {\"source\":\"/opt/algorithm/resources\", \"destination\":\"/opt/algorithm/resources/\"},\n {\"source\":\"/home/algo/.nuget\", \"destination\":\"/home/algo/.nuget/\"}\n \]\n}\n"
+ },
+ {
+ "name":"java11",
+ "display_name":"Java OpenJDK 11.0 (Environments)",
+ "configuration":"{\n \"display_name\": \"Java OpenJDK 11.0 (Environments)\",\n \"artifacts\": \[\n {\"source\":\"/opt/algorithm/target/\*.jar\", \"destination\":\"/opt/algorithm/target/algorithm.jar\"},\n {\"source\":\"/opt/algorithm/target/lib\", \"destination\":\"/opt/algorithm/target/lib/\"}\n \]\n}\n"
+ },
+ {
+ "name":"python2",
+ "display_name":"Python 2.x (Environments)",
+ "configuration":"{\n \"display_name\": \"Python 2.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ {
+ "name":"python3",
+ "display_name":"Python 3.x (Environments)",
+ "configuration":"{\n \"display_name\": \"Python 3.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ {
+ "name":"r36",
+ "display_name":"R 3.6.x (Environments)",
+ "configuration":"{\n \"display_name\": \"R 3.6.x (Environments)\",\n \"req_files\": \[\n \"packages.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"},\n {\"source\":\"/usr/local/lib/R/site-library\", \"destination\":\"/usr/local/lib/R/site-library/\"}\n \]\n}\n\n"
+ },
+ {
+ "name":"scala-2",
+ "display_name":"Scala 2.x & sbt 1.3.x (Environments)",
+ "configuration":"{\n \"display_name\": \"Scala 2.x & sbt 1.3.x (Environments)\",\n \"artifacts\": \[\n {\"source\":\"/opt/algorithm/target/universal/stage\", \"destination\":\"/opt/algorithm/stage/\"}\n \]\n}\n\n"
+ }
+]
+```
+
+#### Listing algorithm environments (REST)
+
+Environment IDs are unique on every cluster, so you'll need to request the list of environments for your language of choice in order to get the ID value associated with a specific environment. To list environments, run the command below, substituting `ALGO_LANG` with a language's `name` value from the output of the command [above](#listing-algorithm-languages-rest). This will print out metadata for the available environments. The response below shows an abridged listing of `python3` environments.
+
+**REST request**
+
+```bash
+$ curl -X GET https://CLUSTER_DOMAIN/v1/algorithm-environments/edge/languages/ALGO_LANG/environments \
+ -H 'Authorization: Simple|Bearer AUTH_TOKEN' \
+ -H 'Content-type: application/json'
+```
+
+**REST response**
+
+```json
+[
+ ...,
+ {
+ "id": "fd980f4f-1f1c-4b2f-a128-d60b40c6567a",
+ "environment_specification_id": "36fd467e-fbfe-4ea6-aa66-df3f403b7132",
+ "display_name": "Python 3.8 + TensorFlow GPU 2.3",
+ "description": "Python 3.8 installation with CUDA 10.1 and TensorFlow 2.3 installed",
+ "created_at": "2021-06-03T19:46:29.111",
+ "language": {
+ "name": "python3",
+ "display_name": "Python 3.x (Environments)",
+ "configuration": "{\n \"display_name\": \"Python 3.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ "machine_type": "GPU"
+ },
+ {
+ "id": "ea8f42fe-ea59-4eab-920a-09176729b4fa",
+ "environment_specification_id": "c6d34b46-0851-455d-97f5-ea8fd8b70552",
+ "display_name": "Python 3.7 + MXNet 1.3.1",
+ "description": "Python 3.7 installation with CUDA 9.0 and MXNet 1.3.1 installed",
+ "created_at": "2021-06-03T18:03:14.704",
+ "language": {
+ "name": "python3",
+ "display_name": "Python 3.x (Environments)",
+ "configuration": "{\n \"display_name\": \"Python 3.x (Environments)\",\n \"req_files\": \[\n \"requirements.txt\"\n \],\n \"artifacts\": \[\n {\"source\":\"/home/algo/.local\", \"destination\":\"/home/algo/.local/\"},\n {\"source\":\"/opt/algorithm\", \"destination\":\"/opt/algorithm/\"}\n \]\n}\n"
+ },
+ "machine_type": "GPU"
+ },
+ ...
+]
+```
+
+Depending on your cluster, the above command might return a very long list of environments, many of which won't be useful to you. We recommend filtering the list returned; for example, to only display environments with TensorFlow 2.x preinstalled, you can pipe the output from above as follows:
+
+```bash
+$ curl ... | grep -iC4 -e "tensorflow 2"
+```
+
+#### Downloading template files (REST)
+
+To download template files, you can use the command below, substituting an `environment_specification_id` for `ENV_SPEC_ID`. Note that no `Authorization` request header is required for this endpoint.
+
+**REST request**
+
+```bash
+$ curl -X GET https://CLUSTER_DOMAIN/v1/algorithm-environments/edge/environment-specifications/ENV_SPEC_ID/template \
+ -o 'template.tar'
+```
+
+Now, unzip the downloaded template files into a target directory `DEST` (you must first create this directory if it doesn't already exist). The files can now be checked in to source control for algorithm development.
+
+```bash
+$ tar -xvf template.tar -C DEST && rm -f template.tar
+```
diff --git a/_pages/platform/algorithm-profile.md b/_pages/platform/algorithm-profile.md
index 036c73804..70900f914 100644
--- a/_pages/platform/algorithm-profile.md
+++ b/_pages/platform/algorithm-profile.md
@@ -33,11 +33,11 @@ For non-enterprise users: If an algorithm charges a royalty, you'll see the amou
### Stars and follows
-At the top-right corner of every algorithm profile page it displays the number of stars and followers an algorithm has.
+At the top-right corner of every algorithm profile page, it displays the number of stars and followers an algorithm has.
Following an algorithm will subscribe you to comments and help you keep up to date with activity on that particular algorithm.
-Starring an algorithm indicates to the developer, as well as other users, that the algorithm is useful to you, and adds it to your starred algorithm list. Your starred algorithm list can be found in the ML service catalog (named **Algorithms** in the screenshot above) on the left nav bar.
+Starring an algorithm indicates to the developer, as well as other users, that the algorithm is useful to you, and adds it to your starred algorithm list. Your starred algorithm list can be found in the ML service catalog (named **Algorithms** in the screenshot above) on the left navbar.
### Overview
@@ -61,7 +61,7 @@ This section includes information about the algorithm's licensing, internet acce
#### Run an example
-In this section there's a test console where you can experiment with the algorithm. Click the **Run Example** button to have the algorithm run on the sample input supplied in the left-hand box, and the results will show in the output box on the right.
+In this section, there's a test console where you can experiment with the algorithm. Click the **Run Example** button to have the algorithm run on the sample input supplied in the left-hand box, and the results will show in the output box on the right.
You can experiment with your own input data, but you'll want to make sure that it's in a format the algorithm accepts. To see details about the proper input format, visit the algorithm's [Docs](#docs) tab.
@@ -69,7 +69,7 @@ You can experiment with your own input data, but you'll want to make sure that i
#### Install and use
-In this section you can toggle between our supported client languages to view code for installing the algo CLI, our command-line based tool, as well as client code to call your algorithm from an external application. To learn about how to call an algorithm, see our [Getting Started Guide]({{site.baseurl}}/algorithm-development/your-first-algo).
+In this section, you can toggle between our supported client languages to view code for installing the algo CLI, our command-line-based tool, as well as client code to call your algorithm from an external application. To learn about how to call an algorithm, see our [Getting Started Guide]({{site.baseurl}}/algorithm-development/your-first-algo).
### Docs
diff --git a/_pages/platform/organization-first-updates.md b/_pages/platform/organization-first-updates.md
new file mode 100644
index 000000000..cce44907b
--- /dev/null
+++ b/_pages/platform/organization-first-updates.md
@@ -0,0 +1,57 @@
+---
+categories: basics
+layout: article
+title: Collaboration-Focused UI
+---
+
+In Algorithmia version 21.2, we've updated our platform with a number of user interface (UI) improvements to support an organizational, collaborative approach to algorithm development and management. It's now easier for teams to work together to deliver MLOps functionality in a shared workspace, while individual users can still utilize their personal workspace as an isolated, "sandbox" environment in order to keep the production environment uncluttered.
+Navigation improvements
+We've relocated resource creation, viewing, and management functionality in an effort to clarify who the owner of a resource is when it's created and at the point it's accessed. In the redesigned interface, you'll now see links in the left-hand navigation menu to navigate directly to workspaces associated with any organizations to which you belong, as well as to your personal workspace. Each resource is scoped within a specific workspace, and in the UI you'll only be able to interact with it from within that workspace.
+Organization workspaces
+Resources (i.e., Algorithms, Data, and API keys) owned by each organization are now visible in that organization's workspace, along with other organization-related tabs that were previously on an organization's profile (i.e., Reporting, Errors, Members, and Invites). Finally, the organization-level settings previously available through the Edit Organization button have been moved to a designated Settings tab.
+
+
+
+Personal workspace
+Analogously, resources owned by individuals are now visible under the individual account's Personal Workspace on the left-hand navigation bar.
+
+
+
+Not only is it now easier to switch between workspaces, we've also updated the workspace landing pages to display the algorithms you've modified most recently so that it's easier to quickly find them and get back to work.
+Org-owned shared resources
+In this org-first version of Algorithmia, only cluster admins can create organizations.
+Furthermore, in addition to algorithms, you can now assign data sources and API keys to be owned by, and automatically shared within, an organization. Org members are automatically granted access to these shared resources, enabling your team to more closely collaborate on algorithm development. Additionally, using org-owned resources, you no longer need to worry about a critical component of a project becoming unavailable or requiring modification when an org member transfers out of the team or leaves the company.
+
+
+
+Org-owned algorithms
+As before, you can create an algorithm under the ownership of an org to enable shared algorithm access between all members of the org. Because there are no longer global resource-creation buttons, to create an algorithm you must navigate to a specific workspace and click the New algorithm button. In the algorithm-creation form, the Owner field will be pre-populated with the name of the org under which you're creating the algorithm. If you'd like to create an algorithm under the ownership of your individual account, you still can; just navigate to your personal workspace and create the algorithm there.
+
+
+
+Org-owned data sources
+In the updated UI, all data sources can now be created under the ownership of a specific org or your individual account. This includes data connectors, which could not previously be shared between members of an org.
+
+
+
+As before, you can also create org-owned data collections to share hosted data between org members.
+
+
+
+Org-owned API keys
+In the updated UI, standard (non-admin) API keys can now be created under the ownership of a specific org instead of just an individual account. Org-owned API keys have access to all resources owned by the org.
+
+
+
+To create a new org-owned API key, navigate to the workspace of the org that you'd like to be the key owner and click the New API key button under the API Keys tab. When you create the key, you'll "assign" it either to the org or to an individual org member account. Algorithmia logs which API key is used for each authenticated resource request, thereby facilitating auditability at the individual user account level while still providing for shared access to resources between multiple individuals.
+Note that as a best practice, we recommend assigning all API keys to individual accounts for auditability and usage reporting purposes. However, there may be cases where it's desirable to assign an org-owned API key to the org itself, so that you can make a key available to all members, so this option is available as well.
+
+
+
+As in the previous UI, we recommend you provide a Label for your key so that it's easily distinguishable from others that you've created and its purpose is clear. The remaining key-creation fields have not changed—you can still control key access on a per-algorithm basis if desired, and you can limit the key's data-access permissions as well.
+
+
+
+User account-owned API keys can still be created by clicking the New API key dropdown in the API Keys tab in your personal workspace. If you have cluster admin permissions, you'll also be able to create an admin API key through this dropdown as well.
+
+
diff --git a/_pages/platform/organization-profile.md b/_pages/platform/organization-profile.md
index 712522117..dda0c2cec 100644
--- a/_pages/platform/organization-profile.md
+++ b/_pages/platform/organization-profile.md
@@ -26,7 +26,7 @@ The following sections cover the basics of using organizations on the Algorithmi
## Profile page contents
-The profile overview provides a rich source of information about an organization. In the overview, you can find the name of an organization, its members, and a list of the algorithms under its ownershipt. If you're an administrator for the organization, you'll also be able to invite and delete members, as well as edit basic information about the organization.
+The profile overview provides a rich source of information about an organization. In the overview, you can find the name of an organization, its members, and a list of the algorithms under its ownership. If you're an administrator for the organization, you'll also be able to invite and delete members, as well as edit basic information about the organization.
@@ -50,7 +50,7 @@ Once your organization has been created you can edit all the provided informatio
### Inviting users to an organization
-You can invite users to your organization by clicking the **Invite Member** button at the upper-right corner of the organization profile. You can invite members either by using their Algorithmia account name or by providing their email address. Users invited by account name will receive an email and as well as an in-platform notification to join the organization.
+You can invite users to your organization by clicking the **Invite Member** button at the upper-right corner of the organization profile. You can invite a member either by using their Algorithmia account name or by providing their email address. Users invited by account name will receive an email and as well as an in-platform notification to join the organization.
If you invite a user that already has an account, then you must invite them using their account name, not their email. Otherwise, they'll receive an email invite to sign up for the platform and join your organization, even though they already have an account.
{: .notice-warning }
@@ -70,13 +70,3 @@ Once members of your organization have created an algorithm and submitted it for
**Note:** Every new version of the algorithm to be published will require independent approval.
-
-At this time you'll be able to set the royalty (if any) if you're using our non-Enterprise platform, and finalize publishing.
-
-
-
-### Checking earnings
-
-If you're using our non-Enterprise platform, to view your organization's earnings and a break down of all algorithms that are called, click on **View Earnings** on your organization's profile.
-
-
diff --git a/_pages/platform/permissions.md b/_pages/platform/permissions.md
index c14f13d5b..58e7d5343 100644
--- a/_pages/platform/permissions.md
+++ b/_pages/platform/permissions.md
@@ -10,7 +10,7 @@ tags: [basics, alg-dev-getting-started, app-dev-getting-started]
title: "Permissions"
---
-Permissions can be found on an algorithm's profile. The algorithm's author can indicate if the algorithm will require internet access, call other algorithms, or if the users can view the source code. Some of these permissions will affect the amount of credits needed to run the algorithm. Be sure to check out the algorithm's permissions section on the description page for more information.
+Permissions can be found on an algorithm's profile. The algorithm's author can indicate if the algorithm will require internet access, call other algorithms, or if the users can view the source code. Navigate to the algorithm's permissions section on the description page for more information.
#### Internet Access
diff --git a/_pages/platform/rbac.md b/_pages/platform/rbac.md
new file mode 100644
index 000000000..ac1088d80
--- /dev/null
+++ b/_pages/platform/rbac.md
@@ -0,0 +1,42 @@
+---
+categories: basics
+layout: article
+title: Role-Based Access Control
+---
+
+On Algorithmia, resource access is controlled using roles and organizations. If you aren't familiar with how roles work on Algorithmia, you can review the glossary definitions for [cluster admin](/developers/glossary#cluster-admin-account), [cluster user](/developers/glossary#user-account), and [org admin](/developers/glossary#org-admin-account).
+
+In the table below, we use the terms **cluster user** and **org member** to differentiate between these roles in the **cluster** and **organization** contexts. However, note that an org member is just any account (cluster admin or cluster user) that's a member of an org. The fact that someone is a cluster admin or cluster user has no effect\* on their permissions to take actions on resources within the org. \*(Because a cluster admin can impersonate other users, including org admins, technically a cluster admin can do anything that an org admin can do. However, this is unrelated to the cluster admin's status in the org; it's just based on their being a cluster admin).
+
+|*(Context)*|*(Cluster)*|*(Cluster)*||*(Organization)*|*(Organization)*|
+|--- |--- |--- |--- |--- |--- |
+|**Role**|**Cluster admin**|**Cluster user**||**Org admin**|**Org member**|
+|||||||
+|Access [browser UI](/developers/glossary#browser-ui)|Y|Y||Y|Y|
+|Access browser UI [admin panel](/developers/glossary#admin-panel)|Y|N||N|N|
+|Create cluster account invite code|Y|N||N|N|
+|Share cluster account invite code|Y|N||N|N|
+|Create [organization](/developers/glossary#org-organization)|Y|Y||Y|Y|
+|Invite user to org|N|N||Y|N|
+|Create [reservation](/developers/glossary#reservation)|Y|N||N|N|
+|View cluster logs|Y|N||N|N|
+|View algorithm error logs|Y|Y (N before v20.5.57)||Y (N before v20.5.57)|Y (N before v20.5.57)|
+|View algorithm build logs|Y (w/ superpowers)|Y (their own algorithms)||Y (org-owned)|Y (org-owned)|
+|Impersonate user|Y|N||N|N|
+|Add sudo user|Y|N||N|N|
+|Create [standard API key](/developers/glossary#standard-api-key)|Y|Y||Y|Y|
+|Create [admin API key](/developers/glossary#admin-api-key)|Y|N||N|N|
+|Upload publicly visible data|Y|Y||Y|Y|
+|Upload privately visible data|Y|Y||Y|Y|
+|Modify data visibility|Y|Y||Y|Y|
+|Create org-owned algorithm|N|N||Y|Y|
+|Create user-owned algorithm|Y|Y||Y|Y|
+|Build/compile algorithm|Y|Y||Y|Y|
+|Publish algorithm publicly|Y|Y||Y|N|
+|Publish algorithm privately|Y|Y||Y|Y|
+|Delete [publicly published algorithm](/developers/glossary#public-algorithm)|Y|N||N|N|
+|Delete [privately published algorithm](/developers/glossary#private-algorithm)|Y|Y||Y|Y|
+|Invoke algorithm|Y|Y||Y|Y|
+|Provide algorithm-invoke access|Y|Y||Y|Y|
+|Modify algorithm source code|Y (their own algorithms)|Y (their own algorithms)||N (user-owned), Y (org-owned)|Y (user-owned), Y (org-owned)|
+|Modify algorithm settings|Y (their own algorithms)|Y (their own algorithms)||N (user-owned), Y (org-owned)|Y (user-owned), Y (org-owned)|
diff --git a/_pages/training-center.md b/_pages/training-center.md
index af5b2f490..6114040b4 100644
--- a/_pages/training-center.md
+++ b/_pages/training-center.md
@@ -11,6 +11,6 @@ show_related: false
title: "Training Center"
---
-Algorithmia's learning management system has been deprecated. Please see DataRobot University for tutorial-based training content to get you up and running with Algorithmia.
+Algorithmia's learning management system has been deprecated. If you're looking for content related to [platform administration](/developers/administration/admin-panel/admin-panel-overview/), it now lives here on the Developer Center and can be found near the bottom of the left-hand navigation panel.
-If you're looking for content related to **platform administration**, it now lives here on the Developer Center and can be found near the bottom of the left-hand navigation panel.
+For tutorial-based training content to get you up and running with DataRobot, please see DataRobot University.
diff --git a/cve_exceptions b/cve_exceptions
index c9c25403b..c9b596ca6 100644
--- a/cve_exceptions
+++ b/cve_exceptions
@@ -1,25 +1,25 @@
CVE-dummy-cve # lines should be formatted with the CVE in the left , followed by a hash then a comment about the date of VP of E approval for the exception
-CVE-2019-18276 # Patch not backported - bash:5.0-4
-CVE-2016-2781 # Patch not backported - coreutils:8.30-3
+CVE-2019-18276 # Patch not backported - bash:5.0-4
+CVE-2016-2781 # Patch not backported - coreutils:8.30-3
CVE-2018-12886 # Patch not backported - gcc-8-base:8.3.0-6
CVE-2019-15847 # Patch not backported - gcc-8-base:8.3.0-6, libgcc1:8.3.0-6, libstdc++6:8.3.0-6
-CVE-2019-14855 # GnuPG 2.0 and above does not use SHA-1 by default therefore are not directly affected by this flaw. - gpgv:2.2.12-1+deb10u1
-CVE-2021-33574 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2021-35942 # Use strtoul instead of atoi so that overflow can be detected - libc-bin:2.28-10, libc6:2.28-10
+CVE-2019-14855 # GnuPG 2.0 and above does not use SHA-1 by default therefore are not directly affected by this flaw. - gpgv:2.2.12-1+deb10u1
+CVE-2021-33574 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2021-35942 # Use strtoul instead of atoi so that overflow can be detected - libc-bin:2.28-10, libc6:2.28-10
CVE-2020-1751 # This flaw did not affect the versions of `glibc` as shipped with Red Hat Enterprise Linux 5, 6, and 7, as they did not include the vulnerable code, which was introduced in a later version of the package.
- - libc-bin:2.28-10, libc6:2.28-10
-CVE-2020-1752 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2021-3326 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2018-20796 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2019-1010022 # maintainer does not consider this a security vulnerability, but rather an enhancement request: https://sourceware.org/bugzilla/show_bug.cgi?id=22850#c3 - libc-bin:2.28-10, libc6:2.28-10
-CVE-2019-1010023 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2019-9192 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2020-6096 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
-CVE-2018-12886 # Patch not backported - libgcc1:8.3.0-6, libstdc++6:8.3.0-6
+ - libc-bin:2.28-10, libc6:2.28-10
+CVE-2020-1752 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2021-3326 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2018-20796 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2019-1010022 # maintainer does not consider this a security vulnerability, but rather an enhancement request: https://sourceware.org/bugzilla/show_bug.cgi?id=22850#c3 - libc-bin:2.28-10, libc6:2.28-10
+CVE-2019-1010023 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2019-9192 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2020-6096 # Patch not backported - libc-bin:2.28-10, libc6:2.28-10
+CVE-2018-12886 # Patch not backported - libgcc1:8.3.0-6, libstdc++6:8.3.0-6
CVE-2018-6829 # Recommends secondary encryption - libgcrypt20:1.8.4-5+deb10u1
-CVE-2019-12290 # Patch not backported - libidn2-0:2.0.5-1+deb10u1
-CVE-2019-17543 # Patch not backported - liblz4-1:1.8.3-1+deb10u1
-CVE-2017-11164 # Patch not backported - libpcre3:2:8.39-12
+CVE-2019-12290 # Patch not backported - libidn2-0:2.0.5-1+deb10u1
+CVE-2019-17543 # Patch not backported - liblz4-1:1.8.3-1+deb10u1
+CVE-2017-11164 # Patch not backported - libpcre3:2:8.39-12
CVE-2017-7245 # Patch not backported - libpcre3:2:8.39-12
CVE-2017-7246 # Patch not backported - libpcre3:2:8.39-12
CVE-2019-20838 # Patch not backported - libpcre3:2:8.39-12
@@ -30,3 +30,10 @@ CVE-2019-19882 # Cannot find patch - login:1:4.5-1.1, passwd:1:4.5-1.1
CVE-2011-4116 # Patch not backported - perl-base:5.28.1-6+deb10u1
CVE-2005-2541 # Red Hat's official stance: "This is the documented and expected behaviour of tar" so this isn't going to be fixed - tar:1.30+dfsg-6
CVE-2019-9923 # Red Hat's official stance: "This is the documented and expected behaviour of tar" so this isn't going to be fixed - tar:1.30+dfsg-6
+CVE-2021-33560 # Patch not backported - libgcrypt20:1.8.4-5+deb10u1
+CVE-2021-43618 # Patch not backported - libgmp10:2:6.1.2+dfsg-4
+CVE-2021-39537 # Response from bug finder: might be a CVE, but the impact should be much lower. There is no code execution. - libncursesw6:6.1+20181013-2+deb10u2, libtinfo6:6.1+20181013-2+deb10u2, ncurses-base:6.1+20181013-2+deb10u2, ncurses-bin:6.1+20181013-2+deb10u2
+CVE-2020-7608 # Recommended to require schema validation and avoid unsafe - yargs-parser:11.1.1
+CVE-2021-33560 # Patch not backported - libgcrypt20:1.8.4-5+deb10u1
+CVE-2021-43618 # Patch not backported - libgmp10:2:6.1.2+dfsg-4
+CVE-2020-16156 # # Recommendation: The first line of protection is to not use untrusted mirrors - perl-base:5.28.1-6+deb10u1
diff --git a/images/post_images/platform/org-first-1627948450822.png b/images/post_images/platform/org-first-1627948450822.png
new file mode 100644
index 000000000..e4d199f7c
Binary files /dev/null and b/images/post_images/platform/org-first-1627948450822.png differ
diff --git a/images/post_images/platform/org-first-1628008092400.png b/images/post_images/platform/org-first-1628008092400.png
new file mode 100644
index 000000000..e38bec4e5
Binary files /dev/null and b/images/post_images/platform/org-first-1628008092400.png differ
diff --git a/images/post_images/platform/org-first-1628008185277.png b/images/post_images/platform/org-first-1628008185277.png
new file mode 100644
index 000000000..cc087599f
Binary files /dev/null and b/images/post_images/platform/org-first-1628008185277.png differ
diff --git a/images/post_images/platform/org-first-1628091096402.png b/images/post_images/platform/org-first-1628091096402.png
new file mode 100644
index 000000000..2b9985a93
Binary files /dev/null and b/images/post_images/platform/org-first-1628091096402.png differ
diff --git a/images/post_images/platform/org-first-1628091205305.png b/images/post_images/platform/org-first-1628091205305.png
new file mode 100644
index 000000000..78b5aab62
Binary files /dev/null and b/images/post_images/platform/org-first-1628091205305.png differ
diff --git a/images/post_images/platform/org-first-1628091274591.png b/images/post_images/platform/org-first-1628091274591.png
new file mode 100644
index 000000000..109799eba
Binary files /dev/null and b/images/post_images/platform/org-first-1628091274591.png differ
diff --git a/images/post_images/platform/org-first-1628091378863.png b/images/post_images/platform/org-first-1628091378863.png
new file mode 100644
index 000000000..6eb9b1774
Binary files /dev/null and b/images/post_images/platform/org-first-1628091378863.png differ
diff --git a/images/post_images/platform/org-first-1628091438177.png b/images/post_images/platform/org-first-1628091438177.png
new file mode 100644
index 000000000..e0fa34b2b
Binary files /dev/null and b/images/post_images/platform/org-first-1628091438177.png differ
diff --git a/images/post_images/platform/org-first-1628091731532.png b/images/post_images/platform/org-first-1628091731532.png
new file mode 100644
index 000000000..c88f51a50
Binary files /dev/null and b/images/post_images/platform/org-first-1628091731532.png differ
diff --git a/images/post_images/platform/org-first-1628091826008.png b/images/post_images/platform/org-first-1628091826008.png
new file mode 100644
index 000000000..8e4585d84
Binary files /dev/null and b/images/post_images/platform/org-first-1628091826008.png differ
diff --git a/package.json b/package.json
index add3f51d0..280e269e3 100644
--- a/package.json
+++ b/package.json
@@ -14,7 +14,7 @@
"reset:cache": "rm -rf .jekyll-cache && rm -rf .jekyll-metadata && rm -rf _site"
},
"devDependencies": {
- "axios": "^0.21.1",
+ "axios": "^0.21.2",
"chai": "^4.3.4",
"http-proxy-middleware": "^0.20.0",
"mocha": "^6.2.0",
diff --git a/yarn.lock b/yarn.lock
index b1cfbbf58..ce25f77cb 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -422,12 +422,12 @@ autoprefixer@^9.8.6:
postcss "^7.0.32"
postcss-value-parser "^4.1.0"
-axios@^0.21.1:
- version "0.21.1"
- resolved "https://registry.yarnpkg.com/axios/-/axios-0.21.1.tgz#22563481962f4d6bde9a76d516ef0e5d3c09b2b8"
- integrity sha512-dKQiRHxGD9PPRIUNIWvZhPTPpl1rf/OxTYKsqKUDjBwYylTvV7SjSHJb9ratfyzM6wCdLCOYLzs73qpg5c4iGA==
+axios@^0.21.2:
+ version "0.21.4"
+ resolved "https://registry.yarnpkg.com/axios/-/axios-0.21.4.tgz#c67b90dc0568e5c1cf2b0b858c43ba28e2eda575"
+ integrity sha512-ut5vewkiu8jjGBdqpM44XxjuCjq9LAKeHVmoVfHVzy8eHgxxq8SbAVQNovDA8mVi05kP0Ea/n/UzcSHcTJQfNg==
dependencies:
- follow-redirects "^1.10.0"
+ follow-redirects "^1.14.0"
bail@^1.0.0:
version "1.0.5"
@@ -1140,10 +1140,10 @@ follow-redirects@^1.0.0:
dependencies:
debug "=3.1.0"
-follow-redirects@^1.10.0:
- version "1.13.1"
- resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.13.1.tgz#5f69b813376cee4fd0474a3aba835df04ab763b7"
- integrity sha512-SSG5xmZh1mkPGyKzjZP8zLjltIfpW32Y5QpdNJyjcfGxK3qo3NDDkZOZSFiGn1A6SclQxY9GzEwAHQ3dmYRWpg==
+follow-redirects@^1.14.0:
+ version "1.14.6"
+ resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.14.6.tgz#8cfb281bbc035b3c067d6cd975b0f6ade6e855cd"
+ integrity sha512-fhUl5EwSJbbl8AR+uYL2KQDxLkdSjZGR36xy46AO7cOMTrCMON6Sa28FmAnC2tRTDbd/Uuzz3aJBv7EBN7JH8A==
foreach@^2.0.4:
version "2.0.5"