Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Motivation

There have been situations where end users need to review a sample, check relevant metrics and trigger a specific Action - for example "for a given sample, IF RapidQC has completed AND LOD>0 THEN submit "FinalAggregation" request in Pipeline's JMS queue". All of these actions to be done/initiated from within Tableau. Beyond visualization, Tableau can't do much else in terms of maintaining a "workflow" in DB, talking to external web-services like JMS and so on. That's why these job is being outsourced to our "SampleAnalyticsWorkflow" web-service - it's written in Scala so there is virtually no limit what tasks it can do for you. Tableau utilize it's UrlAction gadget to make a URL call to SAW service when appropriate.

...

Column nameData typeDescription
PDOVARCHAR2PDO
PDO_SAMPLEVARCHAR2PDO SAMPLE
DATA_TYPEVARCHAR2Exome, WGS
REQUEST_TYPENUMBER
"RequestFinalAggregation", "RequestTopOff", "RequestTopOffGroup", "RequestPoolCreated", "RequestSendToRework", "RequestHoldForTopOff", "RequestWaitingForNewRwData", "RequestReset", "RequestPCHoldForTopOff","RequestVolumeFinal"
1 encodes RequestFinalAggregation, 2 encodes RequestTopOff and so on
USER_NAMEVARCHAR2who triggered this action/transition
PARAMSVARCHAR2This is a free text where Tableau developers can store additional info re this transition which later can be used. Usually it's semicolon-separated-list-of-key=value-pairs
TIMESTAMPDATEwhen this action was triggered
IS_LATESTNUMBERBoolean (1 or 0), Is this the latest transition a given (PDO, PDO_SAMPLE, DATA_TYPE) has gone through. Every time new transition is recorded it comes with IS_LATEST=1 whereas previous latest transition is reset IS_LATEST=0
IP_ADDRESSVARCHAR2where (which IP) this request came from

...

  • requestType is the text-representation of this request (RequestFinalAggregation, RequestTopOff, etc)
  • each individual item is a quintuple representing a single sample like this pdo:pdo_sample:data_type:user_name:parameters (greens are optional)

...

  1. repeat it in the quintuple. Very wasteful.
  2. standalone upfront URL-parameter "&user_name=JohnDoe" (economical and solid). Unfortunately as of now, Tableau's urlAction can't inject the username like this.
  3. 2-step the cookie-dance (economical but might be tricky to use from within Tableau)
    1. record the user_name-cookie in the browser environment by calling this URL http://analytics:8090/api/setCookie?cookies=sampleAnalyticsWorkflow.user_name=JohnDoe (you only need to do this once for your entire session). Cookie/credentials are configured to expire after 10 minutes.
    2. and then make the main SAW call as many times as you need

...

Your environment (browser, OS, versions, etc) implicitly impose a limit on how long a URL can be. On top of that, our web-server provider (Akka-http) also impose limits on what URL-length is acceptable. (Currently akka-http limit is configured to 8194 8192 chars and can possibly be extended further but this should be done rather coservatively since it might lead to more memory consumption). So, best way to answer the question "What URL-size is safe to pass" is to try/test and see. Fortunately, if any truncation happens upstream the last EOURL parameter would definitely not make it and SAW-service can safely detect and report accordingly.

...

  • step1: JOIN cognos.slxre_readgroup_metadata ON (pdo, pdo_sample, data_type) and lookup (research_project_id,collaborator_sample_id) for a given sample
  • step2: send message to appropriate JMS queue

Environments

2 separate environments are provided

artifactProductionDevelopment/Test
tableSAMPLE_ANALYTICS_WORKFLOW SAMPLE_ANALYTICS_WORKFLOW_TEST
web-servicehttp://analytics:8090/api/sampleAnalyticsWorkflowhttp://analytics:8090/api/sampleAnalyticsWorkflowTest
JMS queue

https://picard-jms.broadinstitute.org:8161/admin/browse.jsp?JMSDestination=broad.pushtocloud.enqueue

Pipeline is picking up messages from this queue.

https://picard-jms-dev.broadinstitute.org:8161/admin/browse.jsp?JMSDestination=broad.pushtocloud.analytics

No listener is registered on this queue.

JMS browsehttps://picard-jms.broadinstitute.org:8161/admin/browse.jsphttps://picard-jms-dev.broadinstitute.org:8161/admin/browse.jsp

...