How clients communicate with the Storage Service

There are two ways to communicate with the storage service:

This page discusses:

See Also
Creating Collaborative Widgets Using Storage Service
Storage type scopes
Common operations

The Java client communicates directly with the service at: http://<HOSTNAME>:<BASEPORT+10>/storage-service

The Javascript client communicates with the service through the storage proxy at: http://<HOSTNAME>:<BASEPORT>/mashup-ui/storage

The Javascript client is meant to communicate through the proxy for the 3 following reasons:

  • The XHR requests issued by the Javascript client is subject to the cross-domain restrictions that apply to all XHR requests. Therefore, a proxy on the same port/domain is necessary.

  • The proxy has rudimentary XSRF protection (X-Requested-With header checking) for the Javascript client's calls, preventing a user X to make changes to the state of user Y's data using XSRF.

  • The USER scope. The proxy automatically appends the login name of the user who is currently logged in to the outgoing requests. When communicating directly with the storage service, the user needs to supply the user token manually.

Javascript client use

By default, the Javascript client is included in the Mashup UI. Your custom widget must reference it in its widget.xml file as shown below. 

<Includes>
<Include type="js" path="/resources/javascript/storage/storage-client-0.2.js" />
</Includes>

You must use the StorageService class. 

// Instantiates a new JS storage client
var storage = new StorageClient(storageType, url, options)

The parameters of the StorageClient class are described in the following table.

Parameter

Description

storageType

The value can be user, shared or document depending on the selected scope.

url

The URL of the storage proxy:

http://<HOSTNAME>:<BASEPORT>/mashup-ui/storage

Note that the JS client is only capable of communicating with the proxy, never directly with the storage-service (because of the cross-domain restriction of XHR requests).

If you are in the context of a Mashup UI page, the url parameter is optional. It will be discovered by the storage client automatically.

options

Optional object that accepts any/all of the following properties:

  • timeout: Timeout of the XHR requests in milliseconds (ms) before the error callback is invoked.
  • defaultErrorCallback: function(httpStatusCode, XmlHttpRequestObj, storageErrorEnum, storageErrorEnumDesc): Overrides the default error handler that is invoked on every failed request.
  • defaultSuccessCallback: function(items, XmlHttpRequestObj): Overrides the default success handler that is invoked on every successful request.

All requests are asynchronous. You always need a callback function to read the output from the client's calls: 

// This will not work
// The alert is executed before the response is ever received
var storage = new StorageClient('shared');
var value = storage.get('myStorageKey');
alert(value); // undefined

// This will work better: A callback function is invoked 
// after the client has received the response
storage.get('myStorageKey', function(items) {
    alert(items[0].value); // prints the first item in the collection
});

If the storage service client is instantiated with USER or SHARED, all non-destructive calls (get, getMany, enumerate) take a key parameter, a success and an error callback. 

storage.get(keyString, successCallback, errorCallback);
storage.enumerate(successCallback, errorCallback);
storage.getMany([key1, key2, ...], successCallback, errorCallback);
storage.aggregate([key1, key2, ...],[aggr1, aggr2, ...], successCallback, errorCallback);

The DOCUMENT scope calls take extra parameters: docBuildGroup, docSource, docUrl 

storage.get(docBuildGroup,docSource,docUrl, keyString, successCallback, errorCallback);
storage.enumerate(docBuildGroup,docSource,docUrl, KeyString, successCallback, errorCallback);
storage.enumerate(docBuildGroup,docSource,docUrl, successCallback, errorCallback);

// Gets all the possible permutations for the docObject and key parameters
var docObject1 = {
docBuildGroup : "docBuildGroup",
docSource : "docSource",
docUrl : "docUri"
};

storage.getMany([docObject1, docObject2, ...], [key1, key2, ...], successCallback, errorCallback);
storage.aggregate(docBuildGroup,docSource,docUrl,[key1, key2, ...],[aggr1, aggr2, ...], successCallback,
 errorCallback);

For the USER and SHARED scopes, destructive calls look like this: 

// singleKey is a key targeting a single element (Ex: 'mySingleKey')

// tries to put singleKey -> value in the storage, call errorCallback if 
// key already exists for this scope
storage.put(singleKey, value, successCallback, errorCallback);

// puts singleKey -> value in the storage. If the key already exists, 
// its value is overwritten.
storage.set(singleKey, value, successCallback, errorCallback);

// removes the pair with key singleKey from the storage
storage.del(singleKey, successCallback, errorCallback);


// multiKey is targeting [1:M] values

// appends another value to the bundle pointed to by multiKey
storage.put(multiKey, value, successCallback, errorCallback);
storage.set(multiKey, value, successCallback, errorCallback);

// adds several values to the key and keeps existing ones
storage.putMany(multiKey,[value1,value2, ...], successCallback, errorCallback);
// sets several values to the key and replaces existing ones
storage.setMany(multiKey,[value1,value2, ...], successCallback, errorCallback);


// deletes all values stored for multiKey
storage.del(multiKey, successCallback, errorCallback);


// multiKeyElement is targeting 1 value in a multi valued context
// The only way to get a 'multiKeyElement' is to get all the pairs for a multiKey, 
// and then to use one of the keys in the response

storage.set(multiKeyElement, value, successCallback, errorCallback); 
// updates the existing value 
storage.del(multiKeyElement, successCallback, errorCallback); // deletes one value

For the DOCUMENT scope, destructive calls require:

  • a documentBuildGroup (build group name),

  • a documentSource (source connector name),

  • and a documentId argument. 

    storage.put(documentBuildGroup, documentSource, documentId, singleKey, value, successCallback,
 errorCallback);
storage.set(documentBuildGroup, documentSource, documentId, singleKey, value, successCallback,
 errorCallback);
storage.del(documentBuildGroup, documentSource, documentId, singleKey, successCallback, errorCallback);

// adds several values to the key and keeps existing ones
storage.putMany(documentBuildGroup, documentSource, documentId,multiKey,[value1,value2, ...],
 successCallback, errorCallback);
// sets several values to the key and replaces existing ones
storage.setMany(documentBuildGroup, documentSource, documentId, multiKey,[value1,value2, ...],
 successCallback, errorCallback);

// deletes all pairs stored for a given document:
storage.del(documentBuildGroup, documentSource, documentId, successCallback, errorCallback); 

// ... the multiKey and multiKeyElement examples are like above, but with
//  documentBuildGroup, documentSource, documentId prepended the other parameters

For real examples, go to your <DATADIR>/webapps/mashup-ui/WEB-INF/jsp/widgets/ directory, and look at the source code of the following widgets:

  • savedQueries: which uses multi-valued keys with USER or SHARED storage

  • todoList: which uses single-valued keys with USER or SHARED storage

  • starRating: which uses single-valued keys with DOCUMENT storage.

These widgets make extensive use of the storage service.

Java client use

To use the Java client, make sure that the 360-storage-client.jar is included in your Eclipse project.

All the Java client's calls are synchronous, meaning that they are blocked until a response is received from the server.

The Java client is made to communicate directly with the storage service at:

http://<HOSTNAME>:<BASEPORT+10>/storage-service

The Java client provides very destructive methods (clear operations) therefore it should be used with caution. 

StorageClient client = new StorageClient(Constants.STORAGE_SERVICE_URL, Constants.MASHUPUI_APPID);

client.scratch(); // Scratches everything in the Storage
client.scratchForApplication("default"); // Scratches everything for application 'default'. 
Other applications’ states are preserved
client.scratchForWidget("default", "starRating"); // Scratches all states for the  'starRating' widget
of application 'default'

DocumentClient dclient = client.getDocumentStorage();

//For Cloudview Documents: use a document descriptor(buildgroup,source,docurl) 
DocumentDescriptor descriptor = new DocumentDescriptor("bg0", "docSource", "doc1");

dclient.put(descriptor , "helloWorldKey" "Hello World DocumentStorage!".getBytes("UTF-8")); 
// puts value if key does not already exist

List<byte[]> myValues = new ArrayList<byte[]>();
myValues.add("Hello World DocumentStorage!".getBytes("UTF-8"));
myValues.add("Good bye!".getBytes("UTF-8"));

dclient.putMany(descriptor , "helloWorldKey[]", myValues);
//puts multiple values to the key and keeps existing ones
dclient.setMany(descriptor , "helloWorldKey[]", myValues);
//sets multiple values to the key and replaces existing values

Entry entry = dclient.get(descriptor , "helloWorldKey");
dclient.set(descriptor , "helloWorldKeyDuplicate", entry.getValue()); 
// replaces whatever value (if any) that was previously set in helloWorldKeyDuplicate 

DocumentDescriptor descriptor = new DocumentDescriptor("buildGroup", "source", "document1");
String[] bagKey = new String[] {"testbagkey[]"};
StorageAggregationType[] aggrs = new StorageAggregationType[] { 
				StorageAggregationType.COUNT, 
				StorageAggregationType.AVG,
				StorageAggregationType.MAX,
				StorageAggregationType.MIN,
				StorageAggregationType.SUM
			};

List<AggregationsResult> resultList = this.doc.aggregate(descriptor, bagKey, aggrs);

Example: Simple 'Badge Manager'

On some social networks, a user's profile can be awarded with one or several predefined badges. Badges can be represented as a string of text, and each badge can be either ‘on’ or ‘off’. In the example below, a badge is ‘on’ if the text string is stored for a particular user, and ‘off’ if it does not exist.

The class attaches one or several predefined text strings to a particular Exalead CloudView document with a multi-valued key ending with ‘[]’. No other values than the predefined ones are allowed to exist on the key. 

public class BadgeManager {
                private final String badgeDatabaseKey;
        private final ImmutableSet<String> availableBadges;
        private final DocumentStorage db;
// availableBadges are a list of the predefined text-strings that you want to use, 
// for example ['cool', 'humid', 'warm', 'really-warm']
// badgeTag is the multi-valued key to store on in the storage, for example "perceivedTemperature[]"
        public BadgeManager(String[] availableBadges, String badgeTag) {
                this.availableBadges = ImmutableSet.of(availableBadges);
                this.badgeDatabaseKey = badgeTag;
// Instantiate the Java Storage Client given the URL to the Storage Service 
// (Normally http://CVHOST:[BASEPORT+10]/storage-service) and the Mashup Application ID.
// Instead of .getDocumentStorage() we could have used .getUserStorage() or .getSharedStorage()
                this.db = new StorageClient(Constants.STORAGE_SERVICE_URL,
 Constants.MASHUPUI_APPID).getDocumentStorage();
        }
        public Set<String> getAvailableBadges() {
                return this.availableBadges;
        }
        public void addBadge(DocumentDescriptor document, String badge) throws Exception {
                if (!hasBadgeEntry(componentId, badge)) {
// To put a pair on a document we supply the Document Source and the document id
// The document source is the source connector's name, the document id is the URI of the document
// All values are stored as byte blobs, so the string needs to be converted into a byte[] before submitting
                        db.put(document, badgeDatabaseKey, badge.getBytes("UTF-8"));
                }
        }
        public void removeBadge(DocumentDescriptor document, String badge) throws Exception {
                if (hasBadgeEntry(document, badge)) {
                        Entry toRemove = getBadgeEntry(document, badge);
// To delete a single entry from a multi-valued meta an argument "unique" is needed. 
// Unique is an extra key that identifies a single value in a multi-valued pair
                  deleteByUniqueKey(document, toRemove.getKey().getKey(), toRemove.getKey().getUnique());
                }
        }
        public Set<String> getAllBadgesFor(DocumentDescriptor document) throws Exception {
                Set<String> badges = Sets.newHashSet();
                for (Entry e : getBadgesFromStorage(document)) {
                        badges.add(new String(e.getValue(), "UTF-8"));
                }
                return badges;
        }
        public void removeAllFor(DocumentDescriptor document) throws StorageClientException, Exception {
                // If no unique argument is provided, all of the pair's values are deleted
                db.delete(document, badgeDatabaseKey);
        }
        private boolean hasBadgeEntry(String documentId, String badge) throws Exception {
                return getBadgeEntry(document, badge) != null;
        }

      private Entry getBadgeEntry(DocumentDescriptor document, String badge) throws Exception {
                for (Entry b : getBadgesFromStorage(document)) {
                        if (new String(b.getValue(), "UTF-8").equals(badge)) {
                                return b;
                        }
                }
                return null;
        }
        private List<Entry> getBadgesFromStorage(DocumentDescriptor document) throws Exception {
                // Returns all the values associated with the document 'docId' and the key 'badgeDatabaseKey'
                return db.get(document, badgeDatabaseKey);
        }
}