Downloads / Web Services

Web Services provide a way to enable your program or web site to retrieve DataShop data and insert data back into the central repository programmatically. Our service follows the REST guidelines, which means that requests to web services are done over HTTP using URLs that represent resources.

Web Services API Status

As of October 2018, you can use DataShop web services to:

  • Query the list of datasets in DataShop and get their metadata
  • Query the list of samples for a dataset in DataShop and get their metadata
  • Retrieve transactions
  • Retrieve student-step records
  • Add, retrieve, and delete external analyses
  • Add, retrieve, and modify custom fields at the transaction level
  • Generate a Learning Curve categorization report by dataset and skill model
  • Retrieve a Learning Curve points by dataset, skill model and skill
  • Import a new Knowledge Component Model (KCM) for a dataset

In addition, you can use web services for the following LearnSphere operations:

  • Query the list of workflows in LearnSphere and get their metadata
  • Query the input and output files for a specific workflow

Sample Java Client

We've written a small command-line application in Java that prints the results of URL requests to the console. If you are considering writing an application in Java to interface with DataShop, browse the source for this program, which is included in the download.

Download the Sample Java Client (includes source)

A similar application exists for use with the LearnSphere webservices. This download includes those necessary libraries as well as an example Java client.

Download the LearnSphere Java client and libraries

Documentation for either of these applications

Web Services XML Schema

Use this XML schema file as documentation for the format of responses from DataShop and to validate requests you make to DataShop web services.

DataShop & LearnSphere APIs

Documentation for the application programming interface to DataShop Web Services.

Download the API / User's Guide

Web Services API v0.38 — PDF (631 KB)


Documentation for the application programming interface to LearnSphere Web Services.

Download the API / User's Guide

LearnSphere Web Services API v0.1 — PDF (96 KB)

Sample Java Client Documentation

1. Introduction

DataShop Web Services let you access DataShop data programmatically. By sending a HTTP GET request to a URL, you can get information about datasets and samples in XML format, and retrieve transaction or student-step records in a tab-delimited format. The returned XML is structured according to the pslc_datashop_message.xsd schema.

This package includes a DatashopClient class to make accessing DataShop web services more convenient, described in How to Call a Web Service below.

LearnSphere Web Services allow you to access LearnSphere Workflow data in a similar way. The LearnSphere package includes a LearnSphereWebServicesClient class to make accessing LearnSphere web services more convenient, described in How to Call a Web Service below.

To use web services, you need a DataShop account, a public access key ID to identify you, and a secret key used to authenticate that a request really comes from you. If you don't already have a DataShop account, you can create one for free as described here. Once you have a DataShop account, you can create access keys by visiting this web services page.

2. Package Organization

The DataShop Web Services package is organized in the following manner:

api/: Javadocs for the web services client.
dist/: Contains the datashop-webservices.jar file.
xml/: Contains the DataShop Web Services XML schema.
/: Contains a readme.html file and the webservices.properties template.

3. How to Call a Web Service Programmatically

You can call a web service from your Java program with the DatashopClient class. First, include datashop-webservices.jar on your classpath. Then you can call a web service by following the example in this code snippet.

import edu.cmu.pslc.datashop.webservices.DatashopClient;
...
String apiToken = MY_API_TOKEN;
String secret = MY_SECRET_KEY;
String rootURL = "https://pslcdatashop.web.cmu.edu";
String servicePath = "/datasets/1";

DatashopClient client = new DatashopClient(rootURL, apiToken, secret);

String datasetXml = client.getService(servicePath, "text/xml");

The DatashopClient constructor takes your public API token and secret key as the second and third parameters. The first parameter is the part of the web services URL before "/services". This allows you to make multiple web service calls without having to specify this part of the address every time.

The getService method returns the output of a web service call as a String. In this example, the complete URL being accessed is https://pslcdatashop.web.cmu.edu/services/datasets/1. To access a different web service, use the part that comes after https://pslcdatashop.web.cmu.edu/services in the API documentation as the servicePath argument.

The "text/xml" argument specifies the MIME type format you want to receive. It indicates that the result of this call will be an XML message containing the dataset data. As of now, text/xml is the only valid value for dataset and sample web service requests.

Similarly, you can call a LearnSphere web service from your Java program with the LearnSphereWebServicesClient class. First, include learnsphere-webservices.jar on your classpath. Then you can call a web service by following the example in this code snippet.

import edu.cmu.learnsphere.webservices.LearnSphereWebServicesClient;
...
String apiToken = MY_API_TOKEN;
String secret = MY_SECRET_KEY;
String rootURL = "https://pslcdatashop.web.cmu.edu";
String servicePath = "/workflows";
String fileServicePath = "/workflow_files/1";
Integer workflowId = 1;
String resultZipFilePath = "your_working_dir/result.zip";

LearnSphereWebServicesClient client = new LearnSphereWebServicesClient(rootURL, apiToken, secret);
//get meta data for all worklows you can access
String workflowXml = client.printService(servicePath, "text/xml");
//get single workflow XML
String workflowXml = client.getWorkflow(workflowId);
//get workflow associated files, input and output. the result zip file can be found in resultZipFilePath
client.getFileService(fileServicePath, "application/zip", resultZipFilePath);

4. How to Call a Web Service from the Command Line

You can also access web services directly from the command line. The first step is to edit webservices.properties file in the package root directory to set your public API token and secret key. The initial contents should be

api.token=YOUR_API_TOKEN
secret=YOUR_SECRET_KEY

Replace YOUR_API_TOKEN with your actual token string and YOUR_SECRET_KEY with your actual secret key and save the file. Note that the webservices.properties file must be in the directory from which you will run the program. From the command line, cd to the package root directory. Now you can call a web service like this:

java -jar dist/datashop-webservices.jar "https://pslcdatashop.web.cmu.edu/services/datasets/1"

The results of calling the service will be printed to standard output. The quotes around the URL are recommended because the shell might try to interpret some URL characters. Note also that when calling a web service from the command line, the entire URL must be specified.

Note: If you see an error such as Unable to access jarfile datashop-webservices.jar, make sure your current working directory is the package root containing a "dist" directory and a webservices.properties file.

You can call a LearnSphere web service like this:

java -jar dist/learnsphere-webservices.jar "https://pslcdatashop.web.cmu.edu/learnsphere/services/workflows?verbose=true"
or 
java -jar dist/learnsphere-webservices.jar "https://pslcdatashop.web.cmu.edu/learnsphere/services/workflows/1?verbose=true"
or
//All output files for this workflow is zipped and named "result.zip"; and can be found in the current running directory
java -jar dist/learnsphere-webservices.jar "https://pslcdatashop.web.cmu.edu/learnsphere/services/workflow_files/1?file_type=output"

5. Version Information

To get the version of the jar file, execute the following command on a single line:

java -classpath dist/datashop-webservices.jar
edu.cmu.pslc.datashop.util.VersionInformation

6. Other Programming Languages

Right now, we only have client code for Java. If you would like to access web services from another programming language, please contact us.

Constructing HTTP requests to access the services should be relatively straightforward. Signing each request so that it will properly authenticate you, however, is more complex. We can help you implement the authentication part, and also advise you on constructing valid HTTP requests for accessing web services from your preferred programming language.