ScienceCloud REST Tutorial

Last Updated: Apr 23, 2019 10:27AM PDT

ScienceCloud REST Tutorial

This tutorial describes how to create and access a REST service in ScienceCloud.


The tutorial will take you through all the steps needed to create a ScienceCloud REST service. We will
use the third-party app Postman to illustrate making REST calls. This should give you enough
experience to then implement these calls in a language of choice. These are the steps:

  1. Build a simple protocol on the authoring server that queries ScienceCloud Project data
  2. Save the protocol, which will automatically expose it as a REST web service
  3. In Chrome Postman make REST calls to:
    • Authenticate using basic authentication
    • Call the REST service created above
    • Test with dummy data in the Sandbox environment
  4. Publish the Protocol to the ScienceCloud Production environment
  5. In Chrome Postman make REST calls to:
    • Test your service with Production data

Building a ScienceCloud REST Service Step-by-Step

Build a simple protocol

  1. Request a ScienceCloud authoring license (contact BIOVIA support at
    • Request access to the authoring environment and the ‘Demo’ team. The Demo team will give you access to Demo data for testing your protocol
    • You will be using components from the Project Data collection.
  2. For a detailed description of authoring in ScienceCloud:
  3. We will build a protocol that will retrieve chemistry and assay results stored in demo Project ‘P027’ and pass that back as an SD file.
  4. Connect Pipeline Pilot to the authoring environment and build your protocol:
    • Drag the following components on your canvas:
    • If you are developing an application and prefer to use a ScienceCloud session cookie for authenticatoin, on the ScienceCloud Connection component set authentication to ‘Use Pipeline Pilot Credentials’ and leave username and password blank

      Alternatively, to force users who will run your REST web service to enter their credentials, set authentication to ‘Use Username and Password Parameters’ and leave username and password blank. Right-click on the Username parameter and promote the parameter (name the promoted parameter Username) and right-click on the Password parameter and promote the parameter (name the promoted parameter Password).
    • In the Connection component set the parameter ‘Team’ to ‘Demo’ and promote the parameter
    • In the ‘Project Query’ component set the ‘Project ID’ parameter to ‘P027’ and promote the parameter choosing name 'Project'
    • In the ‘Batch to Assay Results’ set ‘Keep Batch Properties’ to ‘True’
    • In the ‘SD Writer’ component set ‘Destination’ to ‘$(JobDir)\results.sdf’
    • The protocol parameter interface should look like this:
    • Now save the protocol in the following location:
      Protocols/ScienceCloud/<team>/Web Tasks/REST/ScienceCloud Get Batches for a Project With Authentication
    • ‘<team>’ in this case is the team you are working in and generally corresponds to the organization you work for. This is where all the protocols developers in your team build are saved. You can find this information on the ScienceCloud homepage: 
  5. The protocol has been automatically exposed as a REST service. Here is how to access its REST endpoint:
That concludes the Pipeline Pilot part. You can come back to the protocol to extend the interface, for example, to add additional input parameters, or create additional outputs. Each time you do this, save your protocol, and copy a new REST URL if the interface has changed.

Now you are ready to move on to the next step which is testing the service that you build.

Testing the REST Service in Postman

Postman ( is a third-party app that is convenient for testing REST services
before you use them in an application. 
  1. First we issue a call to authenticate ourselves to ScienceCloud. Postman will store our credentials and use that for subsequent REST calls to ScienceCloud:
    • URL:
    • Set Type to ‘Basic Auth’
    • Type your ScienceCloud username and password
    • Click 'Send'
      This will return a lot of information that you can pretty much ignore. The important information is the ‘Key’, and Postman will use this to authenticate from here on. In your own application you will have to extract this key and pass it as a cookie in any subsequent REST calls.
  2. Now we will call our protocol. Paste the URL that you copied from Pipeline Pilot in Postman:<team>/Web Tasks/REST/ScienceCloud Get Batches for a Project With Authentication?$streamdata=*&$format=html&Team=Demo&Project+ID=P027

    Note that ‘<team>’ represents the Team that you are working in, which in most cases is the same as the name of the organization you work for. The Demo Team in this case represent a separate team every ScienceCloud user has access to that gives access to demo data.

    Note that the protocol parameters can be updated here to test other scenarios, for example query other projects or get results back in a different format.

    We are going to make one more improvement. These types of queries can take a while when retrieving a lot of data and the service can time out. It is safer therefore to call the service asynchronously:
  3. We are adding an option ‘$Blocking=0’:<team>/Web Tasks/REST/ScienceCloud Get Batches for a Project With Authentication?$streamdata=*&$format=html&Team=Demo&Project+ID=P027&$Blocking=0 which returns a Job ID.
  4. Use the Job ID to poll for the result (substitute your Job ID):<jobid>
    This returns a status code:
  5. You can retrieve the results when the status is ‘complete’:<jobid>/results/all

    You will typically iterate through the steps of 1) updating your protocol and 2) testing your protocol, until you are happy with the functionality and how it is exposed in the service. You can now focus on parsing these results. The last step then is to deploy your service in the production environment and run it with ‘real’ data.

Publishing your Protocol to Production

Publishing your protocol to the production environment is a somewhat formal process as the production
environment has tight security. The publication wizard in the Pipeline Pilot takes you through this
process step-by-step.
  1. Make sure you have saved your protocol, then right-click the protocol in the explorer tab and select ‘Publish to ScienceCloud’
  2. The next dialog then asks you who you are publishing to. Typically this would be your organization and your collaborators.
  3. The next step is a validation step that will check that your protocol is properly documented, doesn’t violate any security policies, and doesn’t follow any bad protocol writing practices
  4. Once this stage is passed, your protocol is submitted for publication and it will pass through a validation by the ScienceCloud operations team. They will notify you when it is available in the production environment.
  5. Once available, you can call your service. Remember that it is located on a different server, but the path is generally the same.
  6. As in the Sandbox environment, first get a session ID:
  7. For the protocol above then the end point will be:<team>%5CWeb Tasks%5CREST%5CScienceCloud Get Batches for a Project With Authentication&Maximum=1&Project=P027&Team=Demo&$keepJob=1
  • At his point the Production server only supports the launchjob interface
  • ‘$keepjob=1’ is required to prevent the result file from being cleaned up after the protocol runs
  • Use the same process as on the Sandbox to execute the service asynchronously.

Next Steps

You might want to make some more changes based on user feedback of the published protocol and for
that you follow the same process.

Based on the data you are retrieving from ScienceCloud you might want to add more intelligence to your
protocol to write results in a certain way. Pipeline Pilot and the Project Data collection give you a large
arsenal of components to merge, compute statistics, manage property hierarchies, etc.

You might also consider strategies for improving the performance of your protocol by rewriting your
protocol, only returning the required information, and returning it in the most appropriate format (e.g.
file, versus data variable).
seconds ago
a minute ago
minutes ago
an hour ago
hours ago
a day ago
days ago
Invalid characters found