Motivation
There have been situations where end users need to review a sample, check relevant metrics and trigger a specific Action - for example "for a given sample, IF RapidQC has completed AND LOD>0 THEN submit "FinalAggregation" request in Pipeline's JMS queue". All of these actions to be done/initiated from within Tableau. Beyond visualization, Tableau can't do much else in terms of maintaining a "workflow" in DB, talking to external web-services like JMS and so on. That's why these job is being outsourced to our "SampleAnalyticsWorkflow" web-service - it's written in Scala so there is virtually no limit what tasks it can do for you. Tableau utilize it's UrlAction gadget to make a URL call to SAW service when appropriate.
...
| Column name | Data type | Description |
|---|---|---|
| PDO | VARCHAR2 | PDO |
| PDO_SAMPLE | VARCHAR2 | PDO SAMPLE |
| DATA_TYPE | VARCHAR2 | Exome, WGS |
| REQUEST_TYPE | NUMBER | "RequestFinalAggregation", "RequestTopOff", "RequestTopOffGroup", "RequestPoolCreated", "RequestSendToRework", "RequestHoldForTopOff", "RequestWaitingForNewRwData", "RequestReset", "RequestPCHoldForTopOff","RequestVolumeFinal" |
| USER_NAME | VARCHAR2 | who triggered this action/transition |
| PARAMS | VARCHAR2 | This is a free text where Tableau developers can store additional info re this transition which later can be used. Usually it's semicolon-separated-list-of-key=value-pairs |
| TIMESTAMP | DATE | when this action was triggered |
| IS_LATEST | NUMBER | Boolean (1 or 0), Is this the latest transition a given (PDO, PDO_SAMPLE, DATA_TYPE) has gone through. Every time new transition is recorded it comes with IS_LATEST=1 whereas previous latest transition is reset IS_LATEST=0 |
| IP_ADDRESS | VARCHAR2 | where (which IP) this request came from |
...
- requestType is the text-representation of this request (RequestFinalAggregation, RequestTopOff, etc)
- each individual item is a quintuple representing a single sample like this pdo:pdo_sample:data_type:user_name:parameters (greens are optional, trailing colons are not required)
parameters represents any piece of data you might want to attach to a given sample/transition. It is a semicolon-separated-list-of-key=value-pairs. Gets recorded in PARAMS column.
...
- JOIN cognos.slxre_readgroup_metadata ON (pdo, pdo_sample, data_type) and lookup (research_project_id,collaborator_sample_id) for a given sample
- send JSON message to appropriate JMS queue
Environments
2 separate environments are provided
...
SAW extensions
Exome Autopush ETL agent (click for more details)
Manual Push for Final Aggregation using pre-compiled list of samples
In EtlShell: copy/paste comma-separated list of (pdo:pdo_samples:datatype) triplets (note that within each triplet elements are separated by colon)
Code Block language scala import analytics.tiger.agents.SAW CognosDB("analytics.tiger.agents.SAW.PushDev.Manual", SAW.manual("PDO-111:SM-111:E,PDO-222:SM-222:E") flatMap SAW.finalAggDev("nasko"))Make sure you provide appropriate user_name and Dev/Prod labels.
Manual Push for Final Aggregation using SQL to get list of samples
In EtlShell: load (pdo:pdo_samples:datatype) triplets as a single column via SQL
Code Block language scala import analytics.tiger.agents.SAW val sql = """SELECT m.pdo || ':' || m.pdo_sample || ':' || m.data_type data FROM cognos.rapidqc_agg m WHERE m.pdo='PDO-17350' AND m.pdo_sample IN ('SM-I3797','SM-I2NWY','SM-I371P','SM-I2P3C')""" CognosDB("analytics.tiger.agents.SAW.PushDev.viaSQL", SAW.viaSQL(sql) flatMap SAW.finalAggDev("nasko"))