Protocol Development and Publishing Step-by-Step

Last Updated: Dec 04, 2019 05:45AM PST

Summary: This article describes the steps required to publish protocols for use with ScienceCloud.

Covered Topics

Setting Up

All resources for developers can be accessed from the Sandbox Developer Home Page.

You can use the Pipeline Pilot client to work with a local Pipeline Pilot server as well as with the ScienceCloud Authoring server. When creating a new protocol it is common to use a protocol that you saved on a local Pipeline Pilot server as the starting point. When you are ready to test against real data within the target deployment application, move your protocol to the Authoring environment by changing the active server in Pipeline Pilot to the Authoring server. (You may need to install a compatible version of Pipeline Pilot client - you will be prompted if so. A suitable version can be downloaded if needed from the Sandbox Developer home page.)

An easy way to launch Pipeline Pilot client connecting to the ScienceCloud Authoring server is to select the Authoring application tile from the ScienceCloud home page, as shown below.

Alternatively, you can launch Pipeline Pilot client from your desktop and then connect to the Authoring server on client startup. You can also connect to the Authoring server during a client session by selecting the server name in the lower-right corner, and using the Change Active Server Location dialog to switch servers. For the Server Name, type as shown here.

Note: 443 (the port identifier) is optional. This is the standard port for making a secure connection over SSL.

Note: When using the Test feature, be aware that the Ping test is not supported on and This is to be expected for internet-facing cloud servers. The remaining tests should still run and (hopefully) succeed!

Tip: If all of the tests fail, consult your local IT group to determine if there is a local internet proxy server that is blocking access to the remote server on the 443 port.

After connecting to the Authoring server, a Welcome page will be displayed. The interface is no different from connecting to a server on your local network, except that it is customized for the ScienceCloud server, as illustrated below.

Because the Authoring server is a shared resource, you could encounter restrictions beyond those you experience working with Pipeline Pilot on-premise. These restrictions are in place to protect the privacy of all users, including protocol developers. The server rules may place limits on where you can read and write files and also curtail the use of certain components.

You might also notice that other services are not present if they are not required for cloud development. If there is a missing service or facility that you would like to make use of on the Authoring server, contact ScienceCloud Support at, so we can follow-up with your request.

Developing on the Authoring server

A team’s applications in ScienceCloud can be extended and customized using user-provided protocols and components. The Authoring environment provides an authoring environment for creating or editing these protocols and for testing them inside of Pipeline Pilot. They can be tested by publishing them into their target applications. 

Testing your Protocol in the Sandbox Environment

Once your testing in Pipeline Pilot on the authoring server is complete, you can choose to publish your protocol to the Sandbox server. This is done using the Publish to ScienceCloud option from the right-click menu in Pipeline Pilot. This will initiate the publication process.

Selecting a Target Environment

When you initiate the publication process, you first select whether you want to publish the protocol to sandbox or production ScienceCloud. Publishing to the sandbox environment first - for testing - is recommended.

Validating a Protocol

When you initiate the publication process, you first go through a validation step that determines if the protocol is ready for publication. It checks the following features of your protocol:

Validation Step Purpose
Security Verify that there are no security risks with this protocol, (for example, check if the protocol stores passwords that could break certain IT policies).
Performance Verify if the protocol uses any techniques that negatively impact performance.
Design Verify if the protocol incorporates any poor design, (for example, shortcuts that are copies, disabled components).
Documentation Verify that the documentation is complete and meets certain criteria for consistency and clarity.
Style Verify that the protocol meets certain style guidelines.

The validation step can result in any of the following:

  • Errors
  • Warnings
  • Informational messages

Errors will have to be addressed before a protocol can be published. Warnings and informational messages should be addressed as much as possible, but are not required for publishing.

If there are errors or warnings, you can click on the appropriate button and get details.

You can now address any issues in your protocol in Pipeline Pilot and then use the Revalidate feature. This will re-upload your protocol and run the same validation tests again. After you resolve all the issues. clicking Next will submit the protocol for publication.

Publishing a Protocol to Production

Once your testing in the Sandbox is complete, you can choose to publish your protocol to the Production server. This is also done using the Publish to ScienceCloud option from the right-click menu in Pipeline Pilot. This will initiate the publication process.

A dialog will first present the target environments options: sandbox and production. Select production and click the Next button.

Validation follows, and if no errors are found you can click the Next button. The final screen is a confirmation of completion of your publication request.

Click the Done button. The Publish dashboard will be displayed, which shows the status of all of your current and past publication requests.

Protocol Review

Publishing a protocol to Sandbox occurs automatically without manual review, as long as the protocol passes the automated validation tests during the publishing process. For publication to production, after you complete the publication steps your protocol will be sent to the ScienceCloud team for review. This is a manual review that is intended to verify that your protocol is fit for deployment on ScienceCloud. A review will typically be done within two business days. Once completed, you will receive a notification. If accepted, your protocol will be deployed onto the ScienceCloud and will be accessible to your users. If a problem is found, you will receive a "Revision Requested" notification, so you can address the reviewer's comments and resubmit.

Contributing a Notebook Action

A protocol intended to be a Notebook action, previously referred to as a Notebook "task", should be saved to the folder ScienceCloud/Team/Notebook/Tasks, where "Team" is replaced by the name of the ScienceCloud team to which you wish to contribute your protocol as a Notebook action. In order to publish to this location, you need the Notebook privilege called ELNPROTOCOLADMINISTRATION. If you do not have or are not certain if you have that Notebook privilege, contact ScienceCloud Support for assistance.

One additional step is required when publishing a protocol as a Notebook action. Notebook action protocols need to be registered in the Notebook database one time per environment (sandbox and production). After ths initial registration, this registration step is not required for any subsequent updates which you make to that protocol in an environment to which the Notebook action has been registered. You only need to re-publish, but not re-register as Notebook action, your updated protocol to the target environment whenever you modify your protocol and are ready to publish the changes.

For both production and sandbox, the registration of your published protocol as a Notebook action must be performed by ScienceCloud Support. Please contact ScienceCloud Support and ask them to register your protocol as a Notebook action. Provide ScienceCloud Support with the path (folder location including protocol name) of your protocol.

Contributing a Protocol to the Exchange

After publishing a protocol on ScienceCloud, it can be made available to the Pipeline Pilot user community by contributing it to the ScienceCloud Exchange. This is a forum for Pipeline Pilot users who want to experiment with protocols developed by other users. A protocol can be contributed to the Exchange by clicking the blue Share button, as seen in the screenshot above.

Making Changes to a Protocol Publish Request

From the Publish dashboard, in addition to the Share button there are other buttons displayed depending on the status of your publication requests. The Withdraw button enables you to withdraw a request for publishing, prior to a the protocol being published. This will notify the ScienceCloud team to not publish your protocol.

The Unpublish button will notify the ScienceCloud team to delete your published protocol.

The Revert button enables you to revert your published protocol to a previously published version.

Related articles

Contribute Protocols to the Exchange
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
Invalid characters found